2024-09-15T12:07:18.704 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T12:07:18.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T12:07:18.815 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511 branch: wip-matanb-crimson-testing-59543 description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} email: null first_in_suite: false job_id: '7905511' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-testing-59543 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mgr stats period: 30 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore rocksdb cf: false crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 err to stderr: true log flush on exit: true log to file: false log to stderr: true mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: bluestore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd.0: crimson seastar cpu cores: 0-2 osd.1: crimson seastar cpu cores: 3-5 osd.2: crimson seastar cpu cores: 0-2 osd.3: crimson seastar cpu cores: 3-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: crimson sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: min_in: 4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 1236 sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoSiKL10vI0/am/msJYY1PVSH3B5brgaBONgF9qcp2VzYtLoABZ41ns9YCqDFzoyxlada76+GylPlNsWR8/CtU= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPSU+2XUwIEggfuXUdK97f5xygMPd/y8YqscMarSGz8LR1Gw91GjB6iiZhcreUYuazejW9+xoV946ticcwmrMVU= tasks: - install: null - ceph: conf: osd: debug monc: 20 flavor: crimson - thrashosds: ceph_objectstore_tool: false chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_11:59:21 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T12:07:18.815 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa; will attempt to use it 2024-09-15T12:07:18.816 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks 2024-09-15T12:07:18.816 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T12:07:18.817 INFO:teuthology.task.internal:Checking packages... 2024-09-15T12:07:18.841 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '81fedcb3b484c843d22ffbc8d179cb567b7083b5' 2024-09-15T12:07:18.841 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T12:07:18.841 INFO:teuthology.packaging:ref: None 2024-09-15T12:07:18.841 INFO:teuthology.packaging:tag: None 2024-09-15T12:07:18.841 INFO:teuthology.packaging:branch: wip-matanb-crimson-testing-59543 2024-09-15T12:07:18.841 INFO:teuthology.packaging:sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-15T12:07:18.842 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-testing-59543 2024-09-15T12:07:18.982 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4879.g81fedcb3 2024-09-15T12:07:18.983 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T12:07:18.990 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T12:07:18.990 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T12:07:19.005 INFO:teuthology.task.internal:Saving configuration 2024-09-15T12:07:19.021 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T12:07:19.028 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T12:07:19.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 12:00:31.974317', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoSiKL10vI0/am/msJYY1PVSH3B5brgaBONgF9qcp2VzYtLoABZ41ns9YCqDFzoyxlada76+GylPlNsWR8/CtU='} 2024-09-15T12:07:19.075 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 12:00:31.972707', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPSU+2XUwIEggfuXUdK97f5xygMPd/y8YqscMarSGz8LR1Gw91GjB6iiZhcreUYuazejW9+xoV946ticcwmrMVU='} 2024-09-15T12:07:19.075 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T12:07:19.081 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2024-09-15T12:07:19.081 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-15T12:07:19.081 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T12:07:19.141 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc5594ff40>, signals=[15]) 2024-09-15T12:07:19.141 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T12:07:19.148 INFO:teuthology.task.internal:Opening connections... 2024-09-15T12:07:19.149 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:07:19.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:07:19.243 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:07:19.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:07:19.319 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T12:07:19.326 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-15T12:07:19.345 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-15T12:07:19.345 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-15T12:07:19.404 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-15T12:07:19.404 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-15T12:07:19.404 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-15T12:07:19.404 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-15T12:07:19.404 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T12:07:19.405 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T12:07:19.406 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-15T12:07:19.456 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-15T12:07:19.474 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-15T12:07:19.475 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-15T12:07:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T12:07:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T12:07:19.534 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-15T12:07:19.556 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T12:07:19.565 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T12:07:19.572 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T12:07:19.572 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-15T12:07:19.578 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-15T12:07:19.594 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T12:07:19.601 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T12:07:19.601 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-15T12:07:19.632 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-15T12:07:19.747 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T12:07:19.802 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T12:07:19.817 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-09-15T12:07:19.817 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-09-15T12:07:19.817 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T12:07:19.818 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T12:07:19.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:07:19.835 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-15T12:07:19.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:07:19.836 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-15T12:07:19.892 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-15T12:07:19.892 INFO:teuthology.task.kernel:Running kernel on smithi103: 5.14.0-503.el9.x86_64 2024-09-15T12:07:19.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum install -y kernel 2024-09-15T12:07:19.894 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-503.el9.x86_64 2024-09-15T12:07:19.894 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-503.el9.x86_64 2024-09-15T12:07:19.894 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-09-15T12:07:22.024 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-15T12:07:22.411 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-15T12:07:23.717 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-15T12:07:24.229 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-15T12:07:29.250 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-15T12:07:30.346 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-15T12:07:31.684 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-15T12:07:31.797 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 19 kB 00:00 2024-09-15T12:07:32.671 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-15T12:07:32.742 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T12:07:38.051 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-15T12:07:38.390 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-15T12:07:40.141 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T12:07:40.141 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T12:07:40.241 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:07:40.251 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:07:40.252 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-09-15T12:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 79 M 2024-09-15T12:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 126 M 2024-09-15T12:07:40.253 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:07:40.413 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T12:07:40.413 INFO:teuthology.orchestra.run.smithi103.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T12:07:40.495 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:07:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:07:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:Install 4 Packages 2024-09-15T12:07:40.499 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:07:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 79 M 2024-09-15T12:07:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 126 M 2024-09-15T12:07:40.499 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:07:40.586 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 80 kB/s | 23 kB 00:00 2024-09-15T12:07:40.716 INFO:teuthology.orchestra.run.smithi103.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 226 kB/s | 23 kB 00:00 2024-09-15T12:07:41.092 INFO:teuthology.orchestra.run.smithi103.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-15T12:07:41.475 INFO:teuthology.orchestra.run.smithi103.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-15T12:07:41.603 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-15T12:07:41.792 INFO:teuthology.orchestra.run.smithi103.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-15T12:07:41.793 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:07:41.793 INFO:teuthology.orchestra.run.smithi103.stdout:Total 61 MB/s | 79 MB 00:01 2024-09-15T12:07:42.476 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:07:42.486 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 15 MB/s | 29 MB 00:01 2024-09-15T12:07:42.834 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:07:42.834 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:07:43.253 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 12 MB/s | 35 MB 00:02 2024-09-15T12:07:43.254 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:07:43.254 INFO:teuthology.orchestra.run.smithi117.stdout:Total 26 MB/s | 79 MB 00:03 2024-09-15T12:07:43.775 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:07:43.775 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:07:43.936 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:07:44.316 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:07:44.316 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:07:45.266 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:07:45.266 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:07:46.268 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:07:46.925 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T12:07:47.012 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:07:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:07:48.453 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T12:07:48.538 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:07:49.815 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:07:49.947 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:07:51.324 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:07:51.456 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:07:54.777 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:07:54.822 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:07:56.282 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:07:56.331 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:07:59.648 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:01.207 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:37.707 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:37.748 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:38.035 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:38.036 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T12:08:38.036 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:08:38.036 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:08:38.845 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:38.845 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:08:38.845 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:08:38.845 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:38.846 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T12:08:38.846 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T12:08:38.846 INFO:teuthology.orchestra.run.smithi103.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T12:08:38.846 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:08:38.846 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:08:39.206 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T12:08:39.283 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:39.291 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:39.292 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T12:08:39.292 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T12:08:39.292 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T12:08:39.292 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T12:08:39.293 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T12:08:39.293 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T12:08:39.293 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T12:08:39.322 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:39.351 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:39.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T12:08:39.351 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-15T12:08:39.383 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-15T12:08:39.384 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:08:39.385 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T12:08:39.386 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T12:08:39.386 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 5.14.0-505.el9.x86_64 2024-09-15T12:08:39.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T12:08:39.703 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:39.703 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T12:08:39.703 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T12:08:39.703 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T12:08:40.099 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:08:40.585 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:08:40.922 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T12:08:41.011 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:41.012 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T12:08:41.012 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T12:08:41.012 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T12:08:41.012 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T12:08:41.012 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-15T12:08:41.012 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T12:08:41.012 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T12:08:41.071 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T12:08:41.071 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T12:08:41.071 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-09-15T12:08:41.102 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 82.el9 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8274670 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T12:08:41.103 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:08:41.104 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T12:08:41.105 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T12:08:41.105 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-505.el9.x86_64 2024-09-15T12:08:41.105 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T12:08:41.447 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T12:08:41.482 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-15T12:08:41.485 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-15T12:08:41.501 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.TJyUobgnx8 2024-09-15T12:08:41.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TJyUobgnx8 2024-09-15T12:08:41.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.TJyUobgnx8 2024-09-15T12:08:41.709 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.TJyUobgnx8 is 6KB 2024-09-15T12:08:41.734 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-09-15T12:08:41.758 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.TJyUobgnx8 2024-09-15T12:08:41.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T12:08:41.836 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T12:08:41.836 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T12:08:41.836 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T12:08:41.836 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T12:08:41.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T12:08:41.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-15T12:08:43.010 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T12:08:43.049 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-09-15T12:08:43.052 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-09-15T12:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.DPvHRjfuRW 2024-09-15T12:08:43.067 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DPvHRjfuRW 2024-09-15T12:08:43.134 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.DPvHRjfuRW 2024-09-15T12:08:43.279 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.DPvHRjfuRW is 6KB 2024-09-15T12:08:43.327 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.DPvHRjfuRW 2024-09-15T12:08:43.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T12:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T12:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T12:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T12:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T12:08:43.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T12:08:43.553 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-09-15T12:09:12.017 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T12:09:12.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:09:12.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:09:13.556 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-15T12:09:13.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:09:13.557 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:09:30.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-15T12:09:31.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-15T12:09:39.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:09:39.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:09:40.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:09:40.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:09:42.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-15T12:09:44.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-15T12:09:54.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:09:54.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:09:56.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:09:56.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:10:14.465 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-15T12:10:29.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:10:29.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:10:29.854 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-15T12:10:30.215 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:10:30.216 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-505.el9.x86_64"... 2024-09-15T12:10:30.216 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-15T12:10:30.230 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-505.el9.x86_64 2024-09-15T12:10:30.230 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T12:10:30.230 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T12:10:30.231 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T12:10:31.231 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-15T12:10:31.232 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T12:10:31.328 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-09-15T12:10:31.353 DEBUG:teuthology.parallel:result is None 2024-09-15T12:10:54.509 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T12:11:09.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:11:09.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:11:09.839 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-15T12:11:10.189 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:11:10.189 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T12:11:10.189 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-15T12:11:10.205 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-505.el9.x86_64 2024-09-15T12:11:10.205 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T12:11:10.206 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T12:11:10.206 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T12:11:11.206 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T12:11:11.207 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T12:11:11.288 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-09-15T12:11:11.309 DEBUG:teuthology.parallel:result is None 2024-09-15T12:11:11.309 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T12:11:11.316 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T12:11:11.316 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T12:11:11.319 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T12:11:11.339 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T12:11:11.346 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T12:11:11.357 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T12:11:11.357 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T12:11:11.377 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T12:11:11.410 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T12:11:11.420 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T12:11:11.420 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T12:11:11.439 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T12:11:11.476 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T12:11:11.485 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T12:11:11.486 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T12:11:11.497 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T12:11:11.499 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T12:11:11.506 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T12:11:11.506 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T12:11:11.530 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T12:11:11.567 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T12:11:11.576 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T12:11:11.576 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T12:11:11.598 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T12:11:11.625 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T12:11:11.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T12:11:11.744 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T12:11:11.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T12:11:11.866 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:11:11.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T12:11:11.931 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T12:11:11.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T12:11:12.038 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T12:11:12.083 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T12:11:12.126 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:11:12.126 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T12:11:12.195 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-15T12:11:12.198 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-15T12:11:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T12:11:12.282 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T12:11:12.726 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T12:11:12.734 INFO:teuthology.task.internal:Starting timer... 2024-09-15T12:11:12.734 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T12:11:12.758 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T12:11:12.767 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-15T12:11:12.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-15T12:11:12.842 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-15T12:11:12.844 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-15T12:11:12.893 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-15T12:11:12.895 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T12:11:12.895 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-15T12:11:12.917 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-15T12:11:12.918 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-15T12:11:13.009 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-15T12:11:13.010 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-09-15T12:11:13.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T12:11:13.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:11:13.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T12:11:13.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:11:13.064 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T12:11:13.065 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-15T12:11:13.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-15T12:11:13.140 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T12:11:13.150 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T12:11:13.153 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T12:11:13.272 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T12:11:13.284 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T12:11:13.285 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T12:15:29.123 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-09-15T12:15:29.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:15:29.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:15:29.212 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-15T12:15:29.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-15T12:15:29.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:15:29.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T12:15:29.376 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-15T12:15:29.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-15T12:15:29.457 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T12:15:29.471 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T12:15:29.472 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T12:15:29.472 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T12:15:29.475 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T12:15:29.475 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T12:15:29.502 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T12:15:29.516 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T12:15:29.529 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T12:15:29.541 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-15T12:15:29.542 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T12:15:29.552 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-15T12:15:29.566 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-15T12:15:29.567 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T12:15:29.577 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-15T12:15:29.581 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T12:15:29.591 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T12:15:29.605 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T12:15:29.641 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.645 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.665 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-15T12:15:29.668 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T12:15:29.668 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-15T12:15:29.668 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.668 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.668 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.669 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:15:29.669 INFO:teuthology.run_tasks:Running task install... 2024-09-15T12:15:29.678 DEBUG:teuthology.task.install:project ceph 2024-09-15T12:15:29.678 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}} 2024-09-15T12:15:29.678 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'} 2024-09-15T12:15:29.678 INFO:teuthology.task.install:Using flavor: crimson 2024-09-15T12:15:29.685 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T12:15:29.685 INFO:teuthology.task.install:extra packages: [] 2024-09-15T12:15:29.685 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'tag': None, 'wait_for_package': False} 2024-09-15T12:15:29.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-15T12:15:29.688 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'tag': None, 'wait_for_package': False} 2024-09-15T12:15:29.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-15T12:15:29.834 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/ 2024-09-15T12:15:29.834 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4879.g81fedcb3 2024-09-15T12:15:29.847 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/ 2024-09-15T12:15:29.848 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4879.g81fedcb3 2024-09-15T12:15:29.946 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T12:15:29.946 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:15:29.946 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T12:15:29.962 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T12:15:29.962 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:15:29.962 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T12:15:29.973 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T12:15:29.974 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81fedcb3b484c843d22ffbc8d179cb567b7083b5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T12:15:29.989 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T12:15:29.989 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81fedcb3b484c843d22ffbc8d179cb567b7083b5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T12:15:30.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T12:15:30.054 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T12:15:30.115 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T12:15:30.129 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T12:15:30.180 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-15T12:15:30.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-15T12:15:30.197 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-09-15T12:15:30.197 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-09-15T12:15:30.407 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-15T12:15:30.424 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-09-15T12:15:30.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw 2024-09-15T12:15:30.451 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw 2024-09-15T12:15:31.176 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 200 kB/s | 88 kB 00:00 2024-09-15T12:15:31.178 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 193 kB/s | 88 kB 00:00 2024-09-15T12:15:31.487 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-15T12:15:31.492 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 88 kB/s | 24 kB 00:00 2024-09-15T12:15:31.785 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 8.1 kB/s | 2.1 kB 00:00 2024-09-15T12:15:31.788 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 7.8 kB/s | 2.1 kB 00:00 2024-09-15T12:15:32.401 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-15T12:15:33.548 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-15T12:15:35.136 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-15T12:15:36.290 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-15T12:15:39.487 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T12:15:40.808 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 9.7 MB/s | 6.5 MB 00:00 2024-09-15T12:15:41.112 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-15T12:15:42.191 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T12:15:42.375 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 19 kB 00:00 2024-09-15T12:15:43.297 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-15T12:15:47.597 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T12:15:48.695 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-15T12:15:49.948 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:15:49.959 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:15:49.959 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:15:49.959 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 10 M 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.8 M 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.1 M 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 20 M 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 25 k 2024-09-15T12:15:49.960 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 809 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 558 k 2024-09-15T12:15:49.961 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.4 M 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 45 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 143 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 160 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 315 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 302 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 99 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T12:15:49.962 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout:Install 31 Packages 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 2 Packages 2024-09-15T12:15:49.963 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:15:49.964 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 60 M 2024-09-15T12:15:49.964 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:15:51.087 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 10 M 2024-09-15T12:15:51.094 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.8 M 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.1 M 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 20 M 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 25 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 809 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-15T12:15:51.095 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 558 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.4 M 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 45 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 143 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 160 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T12:15:51.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 315 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 302 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 99 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:Install 31 Packages 2024-09-15T12:15:51.097 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 2 Packages 2024-09-15T12:15:51.098 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:15:51.098 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 60 M 2024-09-15T12:15:51.099 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:15:52.506 INFO:teuthology.orchestra.run.smithi117.stdout:(1/33): ceph-base-19.3.0-4879.g81fedcb3.el9.x86 2.7 MB/s | 6.1 MB 00:02 2024-09-15T12:15:52.606 INFO:teuthology.orchestra.run.smithi117.stdout:(2/33): ceph-selinux-19.3.0-4879.g81fedcb3.el9. 251 kB/s | 25 kB 00:00 2024-09-15T12:15:52.790 INFO:teuthology.orchestra.run.smithi117.stdout:(3/33): libcephfs2-19.3.0-4879.g81fedcb3.el9.x8 4.3 MB/s | 809 kB 00:00 2024-09-15T12:15:52.957 INFO:teuthology.orchestra.run.smithi117.stdout:(4/33): libradosstriper1-19.3.0-4879.g81fedcb3. 3.3 MB/s | 558 kB 00:00 2024-09-15T12:15:53.224 INFO:teuthology.orchestra.run.smithi117.stdout:(5/33): ceph-radosgw-19.3.0-4879.g81fedcb3.el9. 3.5 MB/s | 10 MB 00:02 2024-09-15T12:15:53.315 INFO:teuthology.orchestra.run.smithi117.stdout:(6/33): python3-ceph-argparse-19.3.0-4879.g81fe 492 kB/s | 45 kB 00:00 2024-09-15T12:15:53.416 INFO:teuthology.orchestra.run.smithi117.stdout:(7/33): python3-ceph-common-19.3.0-4879.g81fedc 1.4 MB/s | 143 kB 00:00 2024-09-15T12:15:53.549 INFO:teuthology.orchestra.run.smithi117.stdout:(8/33): python3-cephfs-19.3.0-4879.g81fedcb3.el 1.2 MB/s | 160 kB 00:00 2024-09-15T12:15:53.650 INFO:teuthology.orchestra.run.smithi117.stdout:(9/33): python3-rados-19.3.0-4879.g81fedcb3.el9 3.1 MB/s | 315 kB 00:00 2024-09-15T12:15:53.758 INFO:teuthology.orchestra.run.smithi117.stdout:(10/33): python3-rbd-19.3.0-4879.g81fedcb3.el9. 2.7 MB/s | 302 kB 00:00 2024-09-15T12:15:53.851 INFO:teuthology.orchestra.run.smithi117.stdout:(11/33): librgw2-19.3.0-4879.g81fedcb3.el9.x86_ 6.0 MB/s | 5.4 MB 00:00 2024-09-15T12:15:53.876 INFO:teuthology.orchestra.run.smithi117.stdout:(12/33): python3-rgw-19.3.0-4879.g81fedcb3.el9. 849 kB/s | 99 kB 00:00 2024-09-15T12:15:54.001 INFO:teuthology.orchestra.run.smithi117.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 712 kB/s | 107 kB 00:00 2024-09-15T12:15:54.026 INFO:teuthology.orchestra.run.smithi117.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.1 MB/s | 171 kB 00:00 2024-09-15T12:15:54.076 INFO:teuthology.orchestra.run.smithi117.stdout:(15/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-15T12:15:54.102 INFO:teuthology.orchestra.run.smithi117.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-15T12:15:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:(17/33): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-15T12:15:54.202 INFO:teuthology.orchestra.run.smithi117.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-15T12:15:54.228 INFO:teuthology.orchestra.run.smithi117.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 843 kB/s | 42 kB 00:00 2024-09-15T12:15:54.262 INFO:teuthology.orchestra.run.smithi103.stdout:(1/33): ceph-base-19.3.0-4879.g81fedcb3.el9.x86 2.0 MB/s | 6.1 MB 00:03 2024-09-15T12:15:54.379 INFO:teuthology.orchestra.run.smithi103.stdout:(2/33): ceph-radosgw-19.3.0-4879.g81fedcb3.el9. 3.3 MB/s | 10 MB 00:03 2024-09-15T12:15:54.404 INFO:teuthology.orchestra.run.smithi103.stdout:(3/33): ceph-selinux-19.3.0-4879.g81fedcb3.el9. 177 kB/s | 25 kB 00:00 2024-09-15T12:15:54.420 INFO:teuthology.orchestra.run.smithi117.stdout:(20/33): ceph-common-19.3.0-4879.g81fedcb3.el9. 4.8 MB/s | 20 MB 00:04 2024-09-15T12:15:54.445 INFO:teuthology.orchestra.run.smithi117.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 165 kB/s | 36 kB 00:00 2024-09-15T12:15:54.470 INFO:teuthology.orchestra.run.smithi117.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 825 kB/s | 221 kB 00:00 2024-09-15T12:15:54.521 INFO:teuthology.orchestra.run.smithi117.stdout:(23/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-15T12:15:54.521 INFO:teuthology.orchestra.run.smithi103.stdout:(4/33): libcephfs2-19.3.0-4879.g81fedcb3.el9.x8 5.6 MB/s | 809 kB 00:00 2024-09-15T12:15:54.571 INFO:teuthology.orchestra.run.smithi117.stdout:(24/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 314 kB/s | 47 kB 00:00 2024-09-15T12:15:54.596 INFO:teuthology.orchestra.run.smithi117.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 353 kB/s | 26 kB 00:00 2024-09-15T12:15:54.605 INFO:teuthology.orchestra.run.smithi103.stdout:(5/33): libradosstriper1-19.3.0-4879.g81fedcb3. 2.7 MB/s | 558 kB 00:00 2024-09-15T12:15:54.630 INFO:teuthology.orchestra.run.smithi117.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2024-09-15T12:15:54.664 INFO:teuthology.orchestra.run.smithi117.stdout:(27/33): liboath-2.6.7-2.el9.x86_64.rpm 526 kB/s | 49 kB 00:00 2024-09-15T12:15:54.697 INFO:teuthology.orchestra.run.smithi103.stdout:(6/33): python3-ceph-argparse-19.3.0-4879.g81fe 492 kB/s | 45 kB 00:00 2024-09-15T12:15:54.705 INFO:teuthology.orchestra.run.smithi117.stdout:(28/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-15T12:15:54.731 INFO:teuthology.orchestra.run.smithi117.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-15T12:15:54.781 INFO:teuthology.orchestra.run.smithi117.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-15T12:15:54.789 INFO:teuthology.orchestra.run.smithi103.stdout:(7/33): python3-ceph-common-19.3.0-4879.g81fedc 1.5 MB/s | 143 kB 00:00 2024-09-15T12:15:54.857 INFO:teuthology.orchestra.run.smithi117.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-15T12:15:54.898 INFO:teuthology.orchestra.run.smithi103.stdout:(8/33): python3-cephfs-19.3.0-4879.g81fedcb3.el 1.4 MB/s | 160 kB 00:00 2024-09-15T12:15:55.198 INFO:teuthology.orchestra.run.smithi103.stdout:(9/33): librgw2-19.3.0-4879.g81fedcb3.el9.x86_6 8.0 MB/s | 5.4 MB 00:00 2024-09-15T12:15:55.231 INFO:teuthology.orchestra.run.smithi103.stdout:(10/33): python3-rados-19.3.0-4879.g81fedcb3.el 944 kB/s | 315 kB 00:00 2024-09-15T12:15:55.324 INFO:teuthology.orchestra.run.smithi103.stdout:(11/33): python3-rgw-19.3.0-4879.g81fedcb3.el9. 1.1 MB/s | 99 kB 00:00 2024-09-15T12:15:55.357 INFO:teuthology.orchestra.run.smithi103.stdout:(12/33): python3-rbd-19.3.0-4879.g81fedcb3.el9. 1.9 MB/s | 302 kB 00:00 2024-09-15T12:15:55.458 INFO:teuthology.orchestra.run.smithi103.stdout:(13/33): boost-program-options-1.75.0-8.el9.x86 800 kB/s | 107 kB 00:00 2024-09-15T12:15:55.483 INFO:teuthology.orchestra.run.smithi103.stdout:(14/33): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-15T12:15:55.525 INFO:teuthology.orchestra.run.smithi103.stdout:(15/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-15T12:15:55.550 INFO:teuthology.orchestra.run.smithi103.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-09-15T12:15:55.565 INFO:teuthology.orchestra.run.smithi117.stdout:(32/33): librbd1-19.3.0-4879.g81fedcb3.el9.x86_ 4.8 MB/s | 3.8 MB 00:00 2024-09-15T12:15:55.600 INFO:teuthology.orchestra.run.smithi103.stdout:(17/33): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-15T12:15:55.623 INFO:teuthology.orchestra.run.smithi117.stdout:(33/33): librados2-19.3.0-4879.g81fedcb3.el9.x8 3.9 MB/s | 3.5 MB 00:00 2024-09-15T12:15:55.629 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:15:55.629 INFO:teuthology.orchestra.run.smithi117.stdout:Total 11 MB/s | 60 MB 00:05 2024-09-15T12:15:55.634 INFO:teuthology.orchestra.run.smithi103.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-15T12:15:55.668 INFO:teuthology.orchestra.run.smithi103.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 628 kB/s | 42 kB 00:00 2024-09-15T12:15:55.726 INFO:teuthology.orchestra.run.smithi103.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-15T12:15:55.748 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:15:55.760 INFO:teuthology.orchestra.run.smithi103.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 389 kB/s | 36 kB 00:00 2024-09-15T12:15:55.785 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:15:55.785 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:15:55.953 INFO:teuthology.orchestra.run.smithi103.stdout:(22/33): ceph-common-19.3.0-4879.g81fedcb3.el9. 4.1 MB/s | 20 MB 00:04 2024-09-15T12:15:55.978 INFO:teuthology.orchestra.run.smithi103.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 189 kB/s | 47 kB 00:00 2024-09-15T12:15:56.028 INFO:teuthology.orchestra.run.smithi103.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.1 MB/s | 308 kB 00:00 2024-09-15T12:15:56.054 INFO:teuthology.orchestra.run.smithi103.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-09-15T12:15:56.079 INFO:teuthology.orchestra.run.smithi103.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 298 kB/s | 26 kB 00:00 2024-09-15T12:15:56.104 INFO:teuthology.orchestra.run.smithi103.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-15T12:15:56.137 INFO:teuthology.orchestra.run.smithi103.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-15T12:15:56.180 INFO:teuthology.orchestra.run.smithi103.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.1 MB/s | 839 kB 00:00 2024-09-15T12:15:56.230 INFO:teuthology.orchestra.run.smithi103.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-15T12:15:56.322 INFO:teuthology.orchestra.run.smithi103.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-15T12:15:56.457 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:15:56.458 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:15:56.872 INFO:teuthology.orchestra.run.smithi103.stdout:(32/33): librbd1-19.3.0-4879.g81fedcb3.el9.x86_ 5.9 MB/s | 3.8 MB 00:00 2024-09-15T12:15:57.994 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:15:58.022 INFO:teuthology.orchestra.run.smithi103.stdout:(33/33): librados2-19.3.0-4879.g81fedcb3.el9.x8 1.9 MB/s | 3.5 MB 00:01 2024-09-15T12:15:58.028 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:15:58.029 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.7 MB/s | 60 MB 00:06 2024-09-15T12:15:58.148 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:15:58.185 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:15:58.185 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:15:58.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-15T12:15:58.575 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:15:58.675 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:15:58.781 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-15T12:15:58.810 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:15:58.856 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:15:58.856 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:15:58.857 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:15:58.898 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-15T12:15:58.932 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-15T12:15:58.957 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-15T12:15:58.980 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-15T12:15:59.006 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-15T12:15:59.181 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-15T12:15:59.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-15T12:15:59.248 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-15T12:15:59.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 11/35 2024-09-15T12:15:59.321 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-15T12:15:59.410 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-15T12:15:59.434 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-15T12:15:59.522 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-15T12:15:59.567 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-15T12:15:59.688 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-15T12:15:59.718 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-15T12:15:59.759 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-15T12:15:59.791 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-15T12:15:59.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-15T12:15:59.857 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-15T12:16:00.225 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-15T12:16:00.274 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-15T12:16:00.352 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:16:00.365 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-15T12:16:00.404 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 24/35 2024-09-15T12:16:00.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-15T12:16:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-15T12:16:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:16:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:16:01.018 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-15T12:16:01.044 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-15T12:16:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-15T12:16:01.132 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-15T12:16:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:16:01.210 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 28/35 2024-09-15T12:16:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:16:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-15T12:16:01.271 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 29/35 2024-09-15T12:16:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-15T12:16:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-15T12:16:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-15T12:16:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-15T12:16:01.590 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-15T12:16:01.621 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-15T12:16:01.656 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-15T12:16:01.700 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 11/35 2024-09-15T12:16:01.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-15T12:16:01.808 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-15T12:16:01.835 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-15T12:16:01.922 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-15T12:16:01.976 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-15T12:16:02.091 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-15T12:16:02.144 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-15T12:16:02.195 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-15T12:16:02.225 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-15T12:16:02.254 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-15T12:16:02.291 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-15T12:16:02.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-15T12:16:02.701 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-15T12:16:02.792 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-15T12:16:02.822 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 24/35 2024-09-15T12:16:02.859 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-15T12:16:03.470 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-15T12:16:03.504 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-15T12:16:03.559 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-15T12:16:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 28/35 2024-09-15T12:16:03.683 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 29/35 2024-09-15T12:16:03.726 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:04.411 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:04.447 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-15T12:16:04.525 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-15T12:16:04.525 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T12:16:04.525 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T12:16:04.525 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:04.557 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:06.177 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:06.212 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:06.907 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-15T12:16:06.968 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-15T12:16:07.065 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-15T12:16:07.066 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T12:16:07.066 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T12:16:07.066 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:07.098 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:15.107 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:15.107 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-09-15T12:16:15.108 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:16.214 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-15T12:16:16.239 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-15T12:16:16.239 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:16:16.239 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T12:16:16.240 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T12:16:16.240 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T12:16:16.240 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:16.241 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T12:16:16.308 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T12:16:16.308 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:17.271 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:17.271 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-15T12:16:17.271 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-15T12:16:17.271 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-15T12:16:17.272 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-15T12:16:17.272 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-15T12:16:17.272 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-15T12:16:17.272 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-15T12:16:17.272 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:17.618 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 6/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 7/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 8/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 9/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 10/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 11/35 2024-09-15T12:16:17.619 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 13/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-15T12:16:17.620 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-15T12:16:17.621 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-15T12:16:17.621 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-15T12:16:17.621 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-15T12:16:17.621 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-15T12:16:17.621 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-15T12:16:17.622 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-15T12:16:17.622 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:17.622 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-15T12:16:17.622 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/35 2024-09-15T12:16:18.401 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T12:16:18.426 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:18.428 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T12:16:18.481 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.482 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T12:16:18.483 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:18.484 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:16:18.520 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T12:16:18.520 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:18.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-test 2024-09-15T12:16:19.390 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:32 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:16:19.531 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 85 M 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 175 k 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-15T12:16:19.532 INFO:teuthology.orchestra.run.smithi117.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout:Install 9 Packages 2024-09-15T12:16:19.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:16:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 87 M 2024-09-15T12:16:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 797 M 2024-09-15T12:16:19.534 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:16:19.863 INFO:teuthology.orchestra.run.smithi117.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 672 kB/s | 94 kB 00:00 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 6/35 2024-09-15T12:16:19.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 7/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 8/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 9/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 10/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 11/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 13/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-15T12:16:19.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-15T12:16:19.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-15T12:16:19.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-15T12:16:19.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-15T12:16:19.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-15T12:16:19.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-15T12:16:19.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-15T12:16:19.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-15T12:16:19.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-15T12:16:19.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-15T12:16:19.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-15T12:16:19.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/35 2024-09-15T12:16:20.113 INFO:teuthology.orchestra.run.smithi117.stdout:(2/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 2.0 MB/s | 513 kB 00:00 2024-09-15T12:16:20.214 INFO:teuthology.orchestra.run.smithi117.stdout:(3/9): libubsan-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 183 kB 00:00 2024-09-15T12:16:20.372 INFO:teuthology.orchestra.run.smithi117.stdout:(4/9): libcephsqlite-19.3.0-4879.g81fedcb3.el9. 269 kB/s | 175 kB 00:00 2024-09-15T12:16:20.489 INFO:teuthology.orchestra.run.smithi117.stdout:(5/9): protobuf-3.14.0-13.el9.x86_64.rpm 3.7 MB/s | 1.0 MB 00:00 2024-09-15T12:16:20.540 INFO:teuthology.orchestra.run.smithi117.stdout:(6/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T12:16:20.581 INFO:teuthology.orchestra.run.smithi117.stdout:(7/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.9 MB/s | 122 kB 00:00 2024-09-15T12:16:20.607 INFO:teuthology.orchestra.run.smithi117.stdout:(8/9): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:16:21.071 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.072 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:16:21.073 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:21.074 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:16:21.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-test 2024-09-15T12:16:21.972 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:00:33 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:16:22.111 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:16:22.112 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:16:22.112 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:16:22.112 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:16:22.112 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:16:22.112 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 85 M 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 175 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:22.113 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:Install 9 Packages 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 87 M 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 797 M 2024-09-15T12:16:22.114 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:16:22.611 INFO:teuthology.orchestra.run.smithi103.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 522 kB/s | 94 kB 00:00 2024-09-15T12:16:22.786 INFO:teuthology.orchestra.run.smithi103.stdout:(2/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 2.9 MB/s | 513 kB 00:00 2024-09-15T12:16:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:(3/9): libubsan-11.5.0-2.el9.x86_64.rpm 2.1 MB/s | 183 kB 00:00 2024-09-15T12:16:22.978 INFO:teuthology.orchestra.run.smithi103.stdout:(4/9): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-15T12:16:23.062 INFO:teuthology.orchestra.run.smithi103.stdout:(5/9): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-15T12:16:23.112 INFO:teuthology.orchestra.run.smithi103.stdout:(6/9): libcephsqlite-19.3.0-4879.g81fedcb3.el9. 257 kB/s | 175 kB 00:00 2024-09-15T12:16:23.146 INFO:teuthology.orchestra.run.smithi103.stdout:(7/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-09-15T12:16:23.171 INFO:teuthology.orchestra.run.smithi103.stdout:(8/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.0 MB/s | 122 kB 00:00 2024-09-15T12:18:51.557 INFO:teuthology.orchestra.run.smithi103.stdout:(9/9): ceph-test-19.3.0-4879.g81fedcb3.el9.x86_ 581 kB/s | 85 MB 02:29 2024-09-15T12:18:51.558 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:18:51.558 INFO:teuthology.orchestra.run.smithi103.stdout:Total 597 kB/s | 87 MB 02:29 2024-09-15T12:18:51.628 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:18:51.651 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:18:51.651 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:18:52.268 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:18:52.269 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:18:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:18:53.243 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-15T12:18:53.291 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-15T12:18:53.335 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-15T12:18:53.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-15T12:18:53.393 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-15T12:18:53.426 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-15T12:18:53.505 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-15T12:18:53.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-15T12:18:53.578 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-15T12:18:56.234 INFO:teuthology.orchestra.run.smithi117.stdout:(9/9): ceph-test-19.3.0-4879.g81fedcb3.el9.x86_ 554 kB/s | 85 MB 02:36 2024-09-15T12:18:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:18:56.237 INFO:teuthology.orchestra.run.smithi117.stdout:Total 569 kB/s | 87 MB 02:36 2024-09-15T12:18:56.308 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:18:56.330 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:18:56.330 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:18:56.955 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:18:56.955 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:18:57.859 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:18:57.950 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-15T12:18:57.994 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-15T12:18:58.038 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-15T12:18:58.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-15T12:18:58.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-15T12:18:58.120 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-15T12:18:58.193 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-15T12:18:58.238 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-15T12:18:58.264 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-15T12:19:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-15T12:19:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-15T12:19:09.425 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-15T12:19:09.514 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-15T12:19:11.239 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-15T12:19:11.239 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-15T12:19:11.240 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-15T12:19:11.656 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-15T12:19:11.656 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:19:11.656 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:19:11.656 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:19:11.656 INFO:teuthology.orchestra.run.smithi103.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:19:11.657 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:19:12.002 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph 2024-09-15T12:19:12.570 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:03:24 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:19:12.710 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.5 k 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 2.2 M 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 1.6 M 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 291 k 2024-09-15T12:19:12.713 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.1 M 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 70 M 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T12:19:12.714 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T12:19:12.715 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:19:12.716 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:19:12.717 INFO:teuthology.orchestra.run.smithi103.stdout:Install 37 Packages 2024-09-15T12:19:12.717 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:19:12.717 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 84 M 2024-09-15T12:19:12.717 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 897 M 2024-09-15T12:19:12.717 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:19:13.626 INFO:teuthology.orchestra.run.smithi103.stdout:(1/37): ceph-19.3.0-4879.g81fedcb3.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2024-09-15T12:19:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-15T12:19:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-15T12:19:16.156 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T12:19:16.157 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T12:19:16.157 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T12:19:16.157 INFO:teuthology.orchestra.run.smithi117.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-15T12:19:16.157 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:19:16.157 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:19:16.472 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph 2024-09-15T12:19:16.826 INFO:teuthology.orchestra.run.smithi103.stdout:(2/37): ceph-mgr-19.3.0-4879.g81fedcb3.el9.x86_ 460 kB/s | 1.6 MB 00:03 2024-09-15T12:19:17.037 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:03:30 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:19:17.178 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:19:17.180 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.5 k 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 2.2 M 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 1.6 M 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 291 k 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.1 M 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 70 M 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T12:19:17.181 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T12:19:17.182 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T12:19:17.183 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout:Install 37 Packages 2024-09-15T12:19:17.184 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:19:17.185 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 84 M 2024-09-15T12:19:17.185 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 897 M 2024-09-15T12:19:17.185 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:19:17.210 INFO:teuthology.orchestra.run.smithi103.stdout:(3/37): ceph-mds-19.3.0-4879.g81fedcb3.el9.x86_ 572 kB/s | 2.2 MB 00:03 2024-09-15T12:19:17.460 INFO:teuthology.orchestra.run.smithi103.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4879.g81fe 1.1 MB/s | 291 kB 00:00 2024-09-15T12:19:18.010 INFO:teuthology.orchestra.run.smithi103.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 460 kB/s | 253 kB 00:00 2024-09-15T12:19:18.015 INFO:teuthology.orchestra.run.smithi117.stdout:(1/37): ceph-19.3.0-4879.g81fedcb3.el9.x86_64.r 18 kB/s | 6.5 kB 00:00 2024-09-15T12:19:18.311 INFO:teuthology.orchestra.run.smithi103.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 4.1 MB/s | 1.2 MB 00:00 2024-09-15T12:19:18.428 INFO:teuthology.orchestra.run.smithi103.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-09-15T12:19:18.545 INFO:teuthology.orchestra.run.smithi103.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-15T12:19:18.653 INFO:teuthology.orchestra.run.smithi103.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.1 MB/s | 125 kB 00:00 2024-09-15T12:19:18.771 INFO:teuthology.orchestra.run.smithi103.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.8 MB/s | 215 kB 00:00 2024-09-15T12:19:18.946 INFO:teuthology.orchestra.run.smithi103.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-15T12:19:19.011 INFO:teuthology.orchestra.run.smithi103.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-09-15T12:19:19.062 INFO:teuthology.orchestra.run.smithi103.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-09-15T12:19:19.112 INFO:teuthology.orchestra.run.smithi103.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.5 MB/s | 77 kB 00:00 2024-09-15T12:19:19.162 INFO:teuthology.orchestra.run.smithi103.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-15T12:19:19.213 INFO:teuthology.orchestra.run.smithi103.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-15T12:19:19.223 INFO:teuthology.orchestra.run.smithi117.stdout:(2/37): ceph-mgr-19.3.0-4879.g81fedcb3.el9.x86_ 1.0 MB/s | 1.6 MB 00:01 2024-09-15T12:19:19.280 INFO:teuthology.orchestra.run.smithi103.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 334 kB/s | 22 kB 00:00 2024-09-15T12:19:19.321 INFO:teuthology.orchestra.run.smithi103.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-15T12:19:19.347 INFO:teuthology.orchestra.run.smithi103.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-15T12:19:19.372 INFO:teuthology.orchestra.run.smithi103.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-15T12:19:19.414 INFO:teuthology.orchestra.run.smithi103.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 8.4 MB/s | 358 kB 00:00 2024-09-15T12:19:19.431 INFO:teuthology.orchestra.run.smithi103.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 646 kB/s | 11 kB 00:00 2024-09-15T12:19:19.448 INFO:teuthology.orchestra.run.smithi103.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-15T12:19:19.494 INFO:teuthology.orchestra.run.smithi103.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 507 kB/s | 23 kB 00:00 2024-09-15T12:19:19.507 INFO:teuthology.orchestra.run.smithi117.stdout:(3/37): ceph-mds-19.3.0-4879.g81fedcb3.el9.x86_ 1.2 MB/s | 2.2 MB 00:01 2024-09-15T12:19:19.515 INFO:teuthology.orchestra.run.smithi103.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 935 kB/s | 19 kB 00:00 2024-09-15T12:19:19.541 INFO:teuthology.orchestra.run.smithi103.stdout:(26/37): python3-jaraco-text-3.2.0-6.el9.noarch 786 kB/s | 20 kB 00:00 2024-09-15T12:19:19.566 INFO:teuthology.orchestra.run.smithi103.stdout:(27/37): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-15T12:19:19.592 INFO:teuthology.orchestra.run.smithi103.stdout:(28/37): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-15T12:19:19.634 INFO:teuthology.orchestra.run.smithi103.stdout:(29/37): python3-pecan-1.4.2-3.el9.noarch.rpm 6.3 MB/s | 272 kB 00:00 2024-09-15T12:19:19.659 INFO:teuthology.orchestra.run.smithi103.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-15T12:19:19.684 INFO:teuthology.orchestra.run.smithi103.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-15T12:19:19.709 INFO:teuthology.orchestra.run.smithi103.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T12:19:19.735 INFO:teuthology.orchestra.run.smithi103.stdout:(33/37): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-09-15T12:19:19.741 INFO:teuthology.orchestra.run.smithi117.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4879.g81fe 1.2 MB/s | 291 kB 00:00 2024-09-15T12:19:19.768 INFO:teuthology.orchestra.run.smithi103.stdout:(34/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 13 MB/s | 427 kB 00:00 2024-09-15T12:19:19.794 INFO:teuthology.orchestra.run.smithi103.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-15T12:19:20.516 INFO:teuthology.orchestra.run.smithi117.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-15T12:19:21.449 INFO:teuthology.orchestra.run.smithi117.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 1.3 MB/s | 1.2 MB 00:00 2024-09-15T12:19:22.166 INFO:teuthology.orchestra.run.smithi117.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 149 kB/s | 106 kB 00:00 2024-09-15T12:19:22.325 INFO:teuthology.orchestra.run.smithi117.stdout:(8/37): ceph-mon-19.3.0-4879.g81fedcb3.el9.x86_ 1.2 MB/s | 5.1 MB 00:04 2024-09-15T12:19:22.436 INFO:teuthology.orchestra.run.smithi103.stdout:(36/37): ceph-mon-19.3.0-4879.g81fedcb3.el9.x86 594 kB/s | 5.1 MB 00:08 2024-09-15T12:19:22.875 INFO:teuthology.orchestra.run.smithi117.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 191 kB/s | 135 kB 00:00 2024-09-15T12:19:23.025 INFO:teuthology.orchestra.run.smithi117.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 178 kB/s | 125 kB 00:00 2024-09-15T12:19:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-15T12:19:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-15T12:19:23.251 INFO:teuthology.orchestra.run.smithi117.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 1.0 MB/s | 35 kB 00:00 2024-09-15T12:19:23.293 INFO:teuthology.orchestra.run.smithi117.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2024-09-15T12:19:23.327 INFO:teuthology.orchestra.run.smithi117.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-15T12:19:23.360 INFO:teuthology.orchestra.run.smithi117.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-09-15T12:19:23.436 INFO:teuthology.orchestra.run.smithi117.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 298 kB/s | 22 kB 00:00 2024-09-15T12:19:23.487 INFO:teuthology.orchestra.run.smithi117.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-15T12:19:23.512 INFO:teuthology.orchestra.run.smithi117.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-15T12:19:23.538 INFO:teuthology.orchestra.run.smithi117.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-15T12:19:23.580 INFO:teuthology.orchestra.run.smithi117.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 8.4 MB/s | 358 kB 00:00 2024-09-15T12:19:23.605 INFO:teuthology.orchestra.run.smithi117.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-15T12:19:23.630 INFO:teuthology.orchestra.run.smithi117.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 709 kB/s | 18 kB 00:00 2024-09-15T12:19:23.656 INFO:teuthology.orchestra.run.smithi117.stdout:(24/37): python3-urllib3-1.26.5-6.el9.noarch.rp 275 kB/s | 215 kB 00:00 2024-09-15T12:19:23.681 INFO:teuthology.orchestra.run.smithi117.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 472 kB/s | 23 kB 00:00 2024-09-15T12:19:23.715 INFO:teuthology.orchestra.run.smithi117.stdout:(26/37): python3-jaraco-text-3.2.0-6.el9.noarch 784 kB/s | 20 kB 00:00 2024-09-15T12:19:23.740 INFO:teuthology.orchestra.run.smithi117.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 232 kB/s | 19 kB 00:00 2024-09-15T12:19:23.765 INFO:teuthology.orchestra.run.smithi117.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 960 kB/s | 46 kB 00:00 2024-09-15T12:19:23.790 INFO:teuthology.orchestra.run.smithi117.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-15T12:19:23.815 INFO:teuthology.orchestra.run.smithi117.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-15T12:19:23.841 INFO:teuthology.orchestra.run.smithi117.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-15T12:19:23.866 INFO:teuthology.orchestra.run.smithi117.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-15T12:19:23.883 INFO:teuthology.orchestra.run.smithi117.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 855 kB/s | 36 kB 00:00 2024-09-15T12:19:23.908 INFO:teuthology.orchestra.run.smithi117.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 5.4 MB/s | 230 kB 00:00 2024-09-15T12:19:23.925 INFO:teuthology.orchestra.run.smithi117.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-15T12:19:23.950 INFO:teuthology.orchestra.run.smithi117.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 6.2 MB/s | 427 kB 00:00 2024-09-15T12:19:50.213 INFO:teuthology.orchestra.run.smithi117.stdout:(37/37): ceph-osd-19.3.0-4879.g81fedcb3.el9.x86 2.3 MB/s | 70 MB 00:30 2024-09-15T12:19:50.217 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:19:50.217 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.6 MB/s | 84 MB 00:33 2024-09-15T12:19:50.360 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:19:50.402 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:19:50.403 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:19:51.215 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:19:51.216 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:19:52.889 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:19:52.956 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T12:19:52.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T12:19:53.025 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T12:19:53.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T12:19:53.149 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T12:19:53.182 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T12:19:53.274 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T12:19:53.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T12:19:53.393 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T12:19:53.428 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T12:19:53.458 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T12:19:53.492 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T12:19:53.526 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T12:19:53.566 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T12:19:53.609 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T12:19:53.653 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T12:19:53.703 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T12:19:53.744 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T12:19:53.824 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T12:19:53.962 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T12:19:54.051 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T12:19:54.104 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T12:19:54.143 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T12:19:54.192 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T12:19:54.255 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T12:19:54.440 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T12:19:54.497 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T12:19:54.583 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T12:19:54.680 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T12:19:54.716 INFO:teuthology.orchestra.run.smithi103.stdout:(37/37): ceph-osd-19.3.0-4879.g81fedcb3.el9.x86 1.9 MB/s | 70 MB 00:37 2024-09-15T12:19:54.719 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:19:54.720 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.0 MB/s | 84 MB 00:42 2024-09-15T12:19:54.730 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T12:19:54.856 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T12:19:54.858 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:19:54.899 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:19:54.899 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:19:55.049 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 32/37 2024-09-15T12:19:55.073 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T12:19:55.099 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:19:55.687 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:19:55.687 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:19:57.351 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:19:57.419 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T12:19:57.452 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T12:19:57.480 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T12:19:57.551 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T12:19:57.595 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T12:19:57.628 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T12:19:57.720 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T12:19:57.799 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T12:19:57.839 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T12:19:57.883 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T12:19:57.921 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T12:19:57.955 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T12:19:57.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T12:19:58.030 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T12:19:58.073 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T12:19:58.107 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T12:19:58.155 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T12:19:58.187 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T12:19:58.261 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T12:19:58.417 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T12:19:58.496 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T12:19:58.550 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T12:19:58.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T12:19:58.638 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T12:19:58.700 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T12:19:58.897 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T12:19:58.951 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T12:19:59.030 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T12:19:59.118 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T12:19:59.169 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T12:19:59.303 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T12:19:59.495 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 32/37 2024-09-15T12:19:59.527 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-15T12:19:59.553 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-15T12:19:59.553 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:19:59.553 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T12:19:59.553 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T12:19:59.553 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T12:19:59.554 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:05.589 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T12:20:05.627 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:06.190 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T12:20:06.213 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:06.449 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-15T12:20:06.472 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-15T12:20:06.472 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:06.473 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T12:20:06.473 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T12:20:06.473 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T12:20:06.473 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:06.539 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-15T12:20:10.179 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T12:20:10.208 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:10.755 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T12:20:10.778 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:11.014 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-15T12:20:11.037 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-15T12:20:11.037 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:11.038 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T12:20:11.038 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T12:20:11.038 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T12:20:11.038 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:11.101 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-15T12:20:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 6/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T12:20:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T12:20:13.012 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T12:20:13.013 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T12:20:14.138 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T12:20:14.139 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T12:20:14.140 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T12:20:14.141 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:14.142 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:14.409 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-base 2024-09-15T12:20:14.974 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:27 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:15.064 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:20:15.108 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:15.109 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:20:15.109 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:15.177 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install cephadm 2024-09-15T12:20:15.737 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:28 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:15.868 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:15.868 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 785 k 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 785 k 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 791 k 2024-09-15T12:20:15.869 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:16.214 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm-19.3.0-4879.g81fedcb3.el9.noarch.rpm 2.2 MB/s | 785 kB 00:00 2024-09-15T12:20:16.214 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:16.214 INFO:teuthology.orchestra.run.smithi117.stdout:Total 2.2 MB/s | 785 kB 00:00 2024-09-15T12:20:16.215 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:16.220 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:16.220 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:16.226 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:16.226 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:16.296 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:16.775 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:16.867 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:17.254 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:17.515 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:17.744 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 6/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T12:20:17.964 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T12:20:17.965 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T12:20:17.966 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T12:20:18.317 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:31 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:18.455 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 158 k 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:18.456 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:20:18.457 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:18.457 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 158 k 2024-09-15T12:20:18.457 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 644 k 2024-09-15T12:20:18.457 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:18.737 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-immutable-object-cache-19.3.0-4879.g81fedc 566 kB/s | 158 kB 00:00 2024-09-15T12:20:18.738 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:18.738 INFO:teuthology.orchestra.run.smithi117.stdout:Total 563 kB/s | 158 kB 00:00 2024-09-15T12:20:18.738 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:18.747 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:18.747 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:18.804 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:18.804 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T12:20:18.945 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T12:20:18.946 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T12:20:18.947 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T12:20:18.948 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T12:20:18.948 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:18.948 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:18.950 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:19.012 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:19.038 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:19.038 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:19.038 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T12:20:19.038 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:19.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-base 2024-09-15T12:20:19.809 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:31 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:19.898 INFO:teuthology.orchestra.run.smithi103.stdout:Package ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:20:19.940 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:19.941 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:20:19.941 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:19.994 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:19.994 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:19.994 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:19.994 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:19.995 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:19.995 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:20.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install cephadm 2024-09-15T12:20:20.230 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr 2024-09-15T12:20:20.567 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:32 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:20.700 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 785 k 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:20.701 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:20:20.702 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:20.702 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 785 k 2024-09-15T12:20:20.702 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 791 k 2024-09-15T12:20:20.702 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:20.800 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:33 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:20.894 INFO:teuthology.orchestra.run.smithi117.stdout:Package ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:20:20.937 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:20.939 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:20:20.939 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:21.009 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T12:20:21.049 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm-19.3.0-4879.g81fedcb3.el9.noarch.rpm 2.2 MB/s | 785 kB 00:00 2024-09-15T12:20:21.050 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:21.050 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.2 MB/s | 785 kB 00:00 2024-09-15T12:20:21.050 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:21.055 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:21.055 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:21.061 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:21.062 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:21.131 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:21.578 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:34 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:21.621 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:21.704 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:21.711 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:21.712 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:21.712 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 14 M 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 34 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 16 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T12:20:21.713 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:Install 11 Packages 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 18 M 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 155 M 2024-09-15T12:20:21.714 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:22.068 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:22.223 INFO:teuthology.orchestra.run.smithi117.stdout:(1/11): ceph-grafana-dashboards-19.3.0-4879.g81 234 kB/s | 34 kB 00:00 2024-09-15T12:20:22.248 INFO:teuthology.orchestra.run.smithi117.stdout:(2/11): ceph-prometheus-alerts-19.3.0-4879.g81f 96 kB/s | 16 kB 00:00 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:22.303 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:22.415 INFO:teuthology.orchestra.run.smithi117.stdout:(3/11): protobuf-compiler-3.14.0-13.el9.x86_64. 5.1 MB/s | 863 kB 00:00 2024-09-15T12:20:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-15T12:20:22.491 INFO:teuthology.orchestra.run.smithi117.stdout:(5/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.0 MB/s | 269 kB 00:00 2024-09-15T12:20:22.516 INFO:teuthology.orchestra.run.smithi117.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-09-15T12:20:22.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T12:20:22.550 INFO:teuthology.orchestra.run.smithi117.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.3 MB/s | 144 kB 00:00 2024-09-15T12:20:22.575 INFO:teuthology.orchestra.run.smithi117.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.3 MB/s | 31 kB 00:00 2024-09-15T12:20:22.601 INFO:teuthology.orchestra.run.smithi117.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-15T12:20:22.659 INFO:teuthology.orchestra.run.smithi117.stdout:(10/11): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-15T12:20:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:(11/11): ceph-mgr-dashboard-19.3.0-4879.g81fedc 15 MB/s | 14 MB 00:00 2024-09-15T12:20:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:Total 14 MB/s | 18 MB 00:01 2024-09-15T12:20:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:23.091 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:35 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:23.229 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 158 k 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:23.230 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:20:23.231 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:23.231 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 158 k 2024-09-15T12:20:23.231 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 644 k 2024-09-15T12:20:23.231 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:23.323 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:23.323 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:23.474 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-immutable-object-cache-19.3.0-4879.g81fedc 648 kB/s | 158 kB 00:00 2024-09-15T12:20:23.475 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:23.475 INFO:teuthology.orchestra.run.smithi103.stdout:Total 644 kB/s | 158 kB 00:00 2024-09-15T12:20:23.475 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:23.485 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:23.485 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:23.543 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:23.543 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:23.685 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:23.750 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:23.776 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:23.776 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:23.776 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T12:20:23.776 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:23.872 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:23.957 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-15T12:20:23.988 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-15T12:20:24.076 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-15T12:20:24.204 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-15T12:20:24.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-15T12:20:24.333 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-15T12:20:24.393 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-15T12:20:24.430 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-15T12:20:24.462 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 9/11 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:24.747 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:24.960 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr 2024-09-15T12:20:25.535 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:37 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:25.631 INFO:teuthology.orchestra.run.smithi103.stdout:Package ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:20:25.673 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:25.674 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:20:25.674 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:25.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T12:20:26.288 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:38 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:26.421 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:26.422 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:26.422 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:26.422 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 14 M 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 34 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 16 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T12:20:26.423 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:Install 11 Packages 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 18 M 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 155 M 2024-09-15T12:20:26.424 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:27.046 INFO:teuthology.orchestra.run.smithi103.stdout:(1/11): ceph-grafana-dashboards-19.3.0-4879.g81 223 kB/s | 34 kB 00:00 2024-09-15T12:20:27.072 INFO:teuthology.orchestra.run.smithi103.stdout:(2/11): ceph-prometheus-alerts-19.3.0-4879.g81f 92 kB/s | 16 kB 00:00 2024-09-15T12:20:27.213 INFO:teuthology.orchestra.run.smithi103.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-09-15T12:20:27.264 INFO:teuthology.orchestra.run.smithi103.stdout:(4/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-15T12:20:27.297 INFO:teuthology.orchestra.run.smithi103.stdout:(5/11): grpc-data-1.46.7-10.el9.noarch.rpm 587 kB/s | 19 kB 00:00 2024-09-15T12:20:27.373 INFO:teuthology.orchestra.run.smithi103.stdout:(6/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 27 MB/s | 2.0 MB 00:00 2024-09-15T12:20:27.398 INFO:teuthology.orchestra.run.smithi103.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.6 MB/s | 144 kB 00:00 2024-09-15T12:20:27.424 INFO:teuthology.orchestra.run.smithi103.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-15T12:20:27.449 INFO:teuthology.orchestra.run.smithi103.stdout:(9/11): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-15T12:20:27.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 10/11 2024-09-15T12:20:27.674 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-15T12:20:27.741 INFO:teuthology.orchestra.run.smithi103.stdout:(10/11): ceph-mgr-dashboard-19.3.0-4879.g81fedc 17 MB/s | 14 MB 00:00 2024-09-15T12:20:28.075 INFO:teuthology.orchestra.run.smithi103.stdout:(11/11): protobuf-compiler-3.14.0-13.el9.x86_64 861 kB/s | 863 kB 00:01 2024-09-15T12:20:28.078 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:28.078 INFO:teuthology.orchestra.run.smithi103.stdout:Total 11 MB/s | 18 MB 00:01 2024-09-15T12:20:28.111 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:28.131 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:28.131 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:28.389 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:28.390 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:28.943 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:29.032 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-15T12:20:29.061 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-15T12:20:29.153 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-15T12:20:29.289 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-15T12:20:29.348 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-15T12:20:29.442 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-15T12:20:29.492 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-15T12:20:29.521 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-15T12:20:29.553 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 9/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 1/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 2/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 3/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-15T12:20:29.567 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-15T12:20:29.568 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-15T12:20:29.568 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-15T12:20:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-15T12:20:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:29.867 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T12:20:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:29.869 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:30.143 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T12:20:30.702 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:43 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:30.835 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 7.4 M 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T12:20:30.836 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout:Install 12 Packages 2024-09-15T12:20:30.837 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:30.838 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 38 M 2024-09-15T12:20:30.838 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 212 M 2024-09-15T12:20:30.838 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:31.549 INFO:teuthology.orchestra.run.smithi117.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-48 13 MB/s | 7.4 MB 00:00 2024-09-15T12:20:31.799 INFO:teuthology.orchestra.run.smithi117.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 236 kB/s | 191 kB 00:00 2024-09-15T12:20:31.857 INFO:teuthology.orchestra.run.smithi117.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 106 kB/s | 33 kB 00:00 2024-09-15T12:20:31.966 INFO:teuthology.orchestra.run.smithi117.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-09-15T12:20:32.008 INFO:teuthology.orchestra.run.smithi117.stdout:(5/12): libgfortran-11.5.0-2.el9.x86_64.rpm 787 kB/s | 801 kB 00:01 2024-09-15T12:20:32.067 INFO:teuthology.orchestra.run.smithi117.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-15T12:20:32.367 INFO:teuthology.orchestra.run.smithi117.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 816 kB/s | 245 kB 00:00 2024-09-15T12:20:32.742 INFO:teuthology.orchestra.run.smithi117.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-09-15T12:20:32.773 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 10/11 2024-09-15T12:20:32.825 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-15T12:20:32.859 INFO:teuthology.orchestra.run.smithi117.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-15T12:20:33.034 INFO:teuthology.orchestra.run.smithi117.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-09-15T12:20:34.643 INFO:teuthology.orchestra.run.smithi117.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 1.9 MB/s | 4.9 MB 00:02 2024-09-15T12:20:34.762 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-15T12:20:34.762 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 1/11 2024-09-15T12:20:34.762 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 2/11 2024-09-15T12:20:34.762 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 3/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-15T12:20:34.763 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-15T12:20:35.085 INFO:teuthology.orchestra.run.smithi117.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-09-15T12:20:35.086 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:35.086 INFO:teuthology.orchestra.run.smithi117.stdout:Total 9.0 MB/s | 38 MB 00:04 2024-09-15T12:20:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-15T12:20:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:35.095 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T12:20:35.095 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:35.096 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:35.345 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T12:20:35.445 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:35.472 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:35.472 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:35.905 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:35.905 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:35.911 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:04:47 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:36.047 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:36.048 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-15T12:20:36.048 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 7.4 M 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T12:20:36.049 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout:Install 12 Packages 2024-09-15T12:20:36.050 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:36.052 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 38 M 2024-09-15T12:20:36.052 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 212 M 2024-09-15T12:20:36.052 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:36.216 INFO:teuthology.orchestra.run.smithi103.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-15T12:20:36.316 INFO:teuthology.orchestra.run.smithi103.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-09-15T12:20:36.475 INFO:teuthology.orchestra.run.smithi103.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 801 kB 00:00 2024-09-15T12:20:36.550 INFO:teuthology.orchestra.run.smithi103.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 243 kB/s | 18 kB 00:00 2024-09-15T12:20:36.642 INFO:teuthology.orchestra.run.smithi103.stdout:(5/12): ceph-mgr-diskprediction-local-19.3.0-48 14 MB/s | 7.4 MB 00:00 2024-09-15T12:20:36.668 INFO:teuthology.orchestra.run.smithi103.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-15T12:20:36.711 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:36.791 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T12:20:36.810 INFO:teuthology.orchestra.run.smithi103.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-15T12:20:36.833 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T12:20:36.932 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T12:20:36.976 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T12:20:37.159 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T12:20:37.169 INFO:teuthology.orchestra.run.smithi103.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-09-15T12:20:37.191 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T12:20:37.261 INFO:teuthology.orchestra.run.smithi103.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-09-15T12:20:37.267 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T12:20:37.903 INFO:teuthology.orchestra.run.smithi103.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.7 MB/s | 5.1 MB 00:01 2024-09-15T12:20:37.956 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T12:20:37.969 INFO:teuthology.orchestra.run.smithi103.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.7 MB/s | 4.9 MB 00:01 2024-09-15T12:20:38.070 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T12:20:39.324 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T12:20:40.320 INFO:teuthology.orchestra.run.smithi103.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.3 MB/s | 16 MB 00:03 2024-09-15T12:20:40.321 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:40.322 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.9 MB/s | 38 MB 00:04 2024-09-15T12:20:40.352 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T12:20:40.382 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-15T12:20:40.685 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:40.712 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:40.712 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:41.151 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:41.151 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:41.913 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:41.998 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T12:20:42.043 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T12:20:42.141 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T12:20:42.194 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T12:20:42.380 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T12:20:42.410 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T12:20:42.486 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 1/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T12:20:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T12:20:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T12:20:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T12:20:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T12:20:43.149 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T12:20:43.273 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T12:20:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T12:20:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T12:20:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:43.376 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:43.761 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-rook 2024-09-15T12:20:44.333 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:04:57 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:44.466 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:44.467 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 49 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T12:20:44.468 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout:Install 13 Packages 2024-09-15T12:20:44.469 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:44.470 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.2 M 2024-09-15T12:20:44.470 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 26 M 2024-09-15T12:20:44.470 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:44.518 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T12:20:44.765 INFO:teuthology.orchestra.run.smithi117.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 307 kB/s | 26 kB 00:00 2024-09-15T12:20:44.790 INFO:teuthology.orchestra.run.smithi117.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 177 kB/s | 19 kB 00:00 2024-09-15T12:20:44.849 INFO:teuthology.orchestra.run.smithi117.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-09-15T12:20:44.874 INFO:teuthology.orchestra.run.smithi117.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-15T12:20:44.899 INFO:teuthology.orchestra.run.smithi117.stdout:(5/13): ceph-mgr-rook-19.3.0-4879.g81fedcb3.el9 226 kB/s | 49 kB 00:00 2024-09-15T12:20:44.924 INFO:teuthology.orchestra.run.smithi117.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-15T12:20:44.950 INFO:teuthology.orchestra.run.smithi117.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-15T12:20:45.000 INFO:teuthology.orchestra.run.smithi117.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-15T12:20:45.025 INFO:teuthology.orchestra.run.smithi117.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-15T12:20:45.050 INFO:teuthology.orchestra.run.smithi117.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T12:20:45.084 INFO:teuthology.orchestra.run.smithi117.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-15T12:20:45.110 INFO:teuthology.orchestra.run.smithi117.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-15T12:20:45.143 INFO:teuthology.orchestra.run.smithi117.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.2 MB/s | 1.0 MB 00:00 2024-09-15T12:20:45.146 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:45.146 INFO:teuthology.orchestra.run.smithi117.stdout:Total 3.3 MB/s | 2.2 MB 00:00 2024-09-15T12:20:45.207 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:45.222 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:45.223 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:45.310 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:45.313 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:45.552 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T12:20:45.578 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-15T12:20:45.830 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:45.990 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T12:20:46.044 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T12:20:46.099 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T12:20:46.137 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T12:20:46.178 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T12:20:46.279 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T12:20:46.382 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T12:20:46.438 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T12:20:47.145 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T12:20:47.193 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T12:20:47.227 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T12:20:47.275 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T12:20:47.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-15T12:20:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-15T12:20:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 1/13 2024-09-15T12:20:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T12:20:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T12:20:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 1/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T12:20:48.270 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T12:20:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T12:20:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T12:20:48.271 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T12:20:48.373 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:48.374 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:48.644 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T12:20:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T12:20:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:48.661 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:48.661 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T12:20:48.662 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:48.663 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:49.051 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-rook 2024-09-15T12:20:49.205 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:02 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:49.338 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 154 k 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T12:20:49.339 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 7.0 M 2024-09-15T12:20:49.340 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 33 M 2024-09-15T12:20:49.341 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:49.625 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:01 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:49.627 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4879.g81fedcb3.e 748 kB/s | 154 kB 00:00 2024-09-15T12:20:49.661 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-15T12:20:49.711 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-15T12:20:49.728 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 4.8 MB/s | 74 kB 00:00 2024-09-15T12:20:49.754 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 638 kB/s | 58 kB 00:00 2024-09-15T12:20:49.759 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 49 k 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T12:20:49.761 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout:Install 13 Packages 2024-09-15T12:20:49.762 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:49.763 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 2.2 M 2024-09-15T12:20:49.763 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 26 M 2024-09-15T12:20:49.763 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:50.431 INFO:teuthology.orchestra.run.smithi103.stdout:(1/13): ceph-mgr-rook-19.3.0-4879.g81fedcb3.el9 285 kB/s | 49 kB 00:00 2024-09-15T12:20:50.598 INFO:teuthology.orchestra.run.smithi103.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 75 kB/s | 26 kB 00:00 2024-09-15T12:20:50.623 INFO:teuthology.orchestra.run.smithi103.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 52 kB/s | 19 kB 00:00 2024-09-15T12:20:50.870 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-15T12:20:50.872 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:50.872 INFO:teuthology.orchestra.run.smithi117.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-15T12:20:50.942 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:50.952 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:50.953 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:51.015 INFO:teuthology.orchestra.run.smithi103.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 405 kB/s | 159 kB 00:00 2024-09-15T12:20:51.040 INFO:teuthology.orchestra.run.smithi103.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 365 kB/s | 222 kB 00:00 2024-09-15T12:20:51.052 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:51.052 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:51.082 INFO:teuthology.orchestra.run.smithi103.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 772 kB/s | 32 kB 00:00 2024-09-15T12:20:51.107 INFO:teuthology.orchestra.run.smithi103.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 576 kB/s | 279 kB 00:00 2024-09-15T12:20:51.124 INFO:teuthology.orchestra.run.smithi103.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 337 kB/s | 14 kB 00:00 2024-09-15T12:20:51.150 INFO:teuthology.orchestra.run.smithi103.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 398 kB/s | 54 kB 00:00 2024-09-15T12:20:51.192 INFO:teuthology.orchestra.run.smithi103.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-15T12:20:51.217 INFO:teuthology.orchestra.run.smithi103.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-15T12:20:51.234 INFO:teuthology.orchestra.run.smithi103.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 722 kB/s | 59 kB 00:00 2024-09-15T12:20:51.276 INFO:teuthology.orchestra.run.smithi103.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-15T12:20:51.279 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:51.279 INFO:teuthology.orchestra.run.smithi103.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-09-15T12:20:51.350 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:51.358 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:51.366 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:51.366 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:51.455 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:51.456 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:51.526 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T12:20:51.588 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T12:20:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T12:20:52.158 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T12:20:52.190 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T12:20:52.212 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T12:20:52.258 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T12:20:52.275 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T12:20:52.284 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-15T12:20:52.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T12:20:52.354 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T12:20:52.465 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T12:20:52.574 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T12:20:52.631 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/6 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T12:20:53.087 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T12:20:53.371 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T12:20:53.378 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T12:20:53.378 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:53.378 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:53.378 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:53.378 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:53.379 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:53.428 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T12:20:53.462 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T12:20:53.510 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T12:20:53.532 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-15T12:20:53.643 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-fuse 2024-09-15T12:20:54.209 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:07 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:54.342 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:54.342 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:54.342 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:54.342 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:54.342 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 884 k 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout:Install 2 Packages 2024-09-15T12:20:54.343 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:54.344 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 964 k 2024-09-15T12:20:54.344 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 4.1 M 2024-09-15T12:20:54.344 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:54.359 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-15T12:20:54.359 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 1/13 2024-09-15T12:20:54.359 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T12:20:54.359 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T12:20:54.359 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T12:20:54.360 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T12:20:54.691 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T12:20:54.692 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T12:20:54.693 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T12:20:54.693 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T12:20:54.693 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:54.693 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:54.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T12:20:55.013 INFO:teuthology.orchestra.run.smithi117.stdout:(1/2): ceph-fuse-19.3.0-4879.g81fedcb3.el9.x86_ 2.4 MB/s | 884 kB 00:00 2024-09-15T12:20:55.097 INFO:teuthology.orchestra.run.smithi117.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 182 kB/s | 80 kB 00:00 2024-09-15T12:20:55.098 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:55.098 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.2 MB/s | 964 kB 00:00 2024-09-15T12:20:55.110 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:55.135 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:55.135 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:55.216 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:55.217 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:55.400 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:55.514 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:07 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:55.550 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T12:20:55.620 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-15T12:20:55.647 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:55.648 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 154 k 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:55.649 INFO:teuthology.orchestra.run.smithi103.stdout:Install 6 Packages 2024-09-15T12:20:55.650 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:55.650 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 7.0 M 2024-09-15T12:20:55.650 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 33 M 2024-09-15T12:20:55.650 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:20:56.089 INFO:teuthology.orchestra.run.smithi103.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4879.g81fedcb3.e 770 kB/s | 154 kB 00:00 2024-09-15T12:20:56.114 INFO:teuthology.orchestra.run.smithi103.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-15T12:20:56.181 INFO:teuthology.orchestra.run.smithi103.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.8 MB/s | 548 kB 00:00 2024-09-15T12:20:56.206 INFO:teuthology.orchestra.run.smithi103.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 627 kB/s | 58 kB 00:00 2024-09-15T12:20:56.240 INFO:teuthology.orchestra.run.smithi103.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-15T12:20:56.310 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-15T12:20:56.310 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/2 2024-09-15T12:20:56.449 INFO:teuthology.orchestra.run.smithi103.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-15T12:20:56.451 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:56.451 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.8 MB/s | 7.0 MB 00:00 2024-09-15T12:20:56.520 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:20:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:20:56.531 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:56.590 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:56.631 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:20:56.631 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:20:56.893 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-volume 2024-09-15T12:20:56.938 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:20:57.112 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T12:20:57.158 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T12:20:57.466 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:10 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:20:57.603 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 285 k 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:20:57.604 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:20:57.605 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:57.605 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 285 k 2024-09-15T12:20:57.605 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 1.3 M 2024-09-15T12:20:57.605 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:20:57.673 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T12:20:57.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T12:20:57.827 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-volume-19.3.0-4879.g81fedcb3.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-15T12:20:57.827 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:20:57.828 INFO:teuthology.orchestra.run.smithi117.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-15T12:20:57.828 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:20:57.837 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:20:57.837 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:20:57.837 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T12:20:57.863 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-15T12:20:57.869 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:20:57.869 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:20:58.097 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:20:58.170 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:58.188 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:58.188 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:20:58.188 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T12:20:58.188 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:58.688 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-15T12:20:58.688 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/6 2024-09-15T12:20:58.688 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T12:20:58.688 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T12:20:58.689 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T12:20:58.689 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T12:20:58.957 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T12:20:58.957 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:58.957 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:20:58.957 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:58.957 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:58.958 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:20:59.202 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:20:59.204 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-fuse 2024-09-15T12:20:59.489 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados-devel 2024-09-15T12:20:59.769 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:11 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:20:59.902 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:20:59.902 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:59.902 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:20:59.902 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 884 k 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout:Install 2 Packages 2024-09-15T12:20:59.903 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:20:59.904 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 964 k 2024-09-15T12:20:59.904 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 4.1 M 2024-09-15T12:20:59.904 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:00.071 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:13 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:00.144 INFO:teuthology.orchestra.run.smithi103.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 409 kB/s | 80 kB 00:00 2024-09-15T12:21:00.203 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 125 k 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:00.204 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 125 k 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 445 k 2024-09-15T12:21:00.205 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:21:00.261 INFO:teuthology.orchestra.run.smithi103.stdout:(2/2): ceph-fuse-19.3.0-4879.g81fedcb3.el9.x86_ 2.8 MB/s | 884 kB 00:00 2024-09-15T12:21:00.261 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:00.261 INFO:teuthology.orchestra.run.smithi103.stdout:Total 2.6 MB/s | 964 kB 00:00 2024-09-15T12:21:00.273 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:00.296 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:00.296 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:00.377 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:00.377 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:00.389 INFO:teuthology.orchestra.run.smithi117.stdout:librados-devel-19.3.0-4879.g81fedcb3.el9.x86_64 678 kB/s | 125 kB 00:00 2024-09-15T12:21:00.390 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:00.390 INFO:teuthology.orchestra.run.smithi117.stdout:Total 672 kB/s | 125 kB 00:00 2024-09-15T12:21:00.390 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:21:00.396 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:21:00.396 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:21:00.460 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:21:00.461 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:21:00.564 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:00.632 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:21:00.705 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T12:21:00.720 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:00.775 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-15T12:21:01.169 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:01.405 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-15T12:21:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/2 2024-09-15T12:21:01.648 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs2 2024-09-15T12:21:01.757 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T12:21:01.757 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:01.757 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:01.758 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T12:21:01.758 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:01.758 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:02.032 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-volume 2024-09-15T12:21:02.225 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:15 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:02.316 INFO:teuthology.orchestra.run.smithi117.stdout:Package libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:02.362 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:02.363 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:02.363 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:02.433 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install libcephfs-devel 2024-09-15T12:21:02.608 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:14 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:02.751 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:02.751 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:02.751 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 285 k 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 285 k 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 1.3 M 2024-09-15T12:21:02.752 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:02.959 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-volume-19.3.0-4879.g81fedcb3.el9.noarch.rp 1.4 MB/s | 285 kB 00:00 2024-09-15T12:21:02.959 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:02.959 INFO:teuthology.orchestra.run.smithi103.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-15T12:21:02.960 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:02.968 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:02.968 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:02.997 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:15 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:03.000 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:03.000 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:03.131 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:03.131 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:03.131 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 32 k 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:21:03.132 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:03.133 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 32 k 2024-09-15T12:21:03.133 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 142 k 2024-09-15T12:21:03.133 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:21:03.215 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:03.276 INFO:teuthology.orchestra.run.smithi117.stdout:libcephfs-devel-19.3.0-4879.g81fedcb3.el9.x86_6 221 kB/s | 32 kB 00:00 2024-09-15T12:21:03.277 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:03.277 INFO:teuthology.orchestra.run.smithi117.stdout:Total 219 kB/s | 32 kB 00:00 2024-09-15T12:21:03.277 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:21:03.281 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:21:03.281 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:21:03.286 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:21:03.297 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:21:03.298 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:21:03.302 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:21:03.302 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:21:03.302 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T12:21:03.302 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:03.388 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:21:03.490 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:03.895 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:04.134 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:04.134 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:04.135 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:21:04.135 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:04.135 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:04.135 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:04.323 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librados2 2024-09-15T12:21:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-15T12:21:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:04.348 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-15T12:21:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:04.348 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:04.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librados-devel 2024-09-15T12:21:04.889 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:17 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:04.979 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:05.022 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:05.023 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:05.023 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:05.102 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install librbd1 2024-09-15T12:21:05.216 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:17 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:05.355 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 125 k 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:05.356 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:05.357 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 125 k 2024-09-15T12:21:05.357 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 445 k 2024-09-15T12:21:05.357 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:05.558 INFO:teuthology.orchestra.run.smithi103.stdout:librados-devel-19.3.0-4879.g81fedcb3.el9.x86_64 620 kB/s | 125 kB 00:00 2024-09-15T12:21:05.559 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:05.559 INFO:teuthology.orchestra.run.smithi103.stdout:Total 614 kB/s | 125 kB 00:00 2024-09-15T12:21:05.559 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:05.565 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:05.565 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:05.632 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:05.632 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:05.670 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:18 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:05.760 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:05.802 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:05.804 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:05.805 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:05.805 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:05.889 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rados 2024-09-15T12:21:05.900 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:06.402 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:06.451 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:19 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:06.540 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:06.583 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:06.584 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:06.584 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:06.655 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rgw 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:06.658 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:06.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install libcephfs2 2024-09-15T12:21:07.220 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:20 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:07.311 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:07.354 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:07.355 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:07.355 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:07.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-cephfs 2024-09-15T12:21:07.445 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:19 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:07.537 INFO:teuthology.orchestra.run.smithi103.stdout:Package libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:07.581 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:07.582 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:07.582 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:07.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install libcephfs-devel 2024-09-15T12:21:07.995 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:20 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:08.085 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:08.128 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:08.129 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:08.129 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:08.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install python3-rbd 2024-09-15T12:21:08.262 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:20 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:08.396 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 32 k 2024-09-15T12:21:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 32 k 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 142 k 2024-09-15T12:21:08.398 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:08.555 INFO:teuthology.orchestra.run.smithi103.stdout:libcephfs-devel-19.3.0-4879.g81fedcb3.el9.x86_6 202 kB/s | 32 kB 00:00 2024-09-15T12:21:08.555 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:08.556 INFO:teuthology.orchestra.run.smithi103.stdout:Total 201 kB/s | 32 kB 00:00 2024-09-15T12:21:08.556 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:08.559 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:08.559 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:08.579 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:08.579 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:08.794 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:08.806 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:21 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:08.897 INFO:teuthology.orchestra.run.smithi117.stdout:Package python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:08.941 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:08.942 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2024-09-15T12:21:08.943 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:09.014 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-fuse 2024-09-15T12:21:09.187 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:09.431 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:09.578 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:22 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:09.627 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librados2 2024-09-15T12:21:09.710 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:09.711 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:09.711 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-15T12:21:09.711 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:09.711 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:21:09.711 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 91 k 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 91 k 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 321 k 2024-09-15T12:21:09.712 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:21:09.904 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-fuse-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 474 kB/s | 91 kB 00:00 2024-09-15T12:21:09.905 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:09.905 INFO:teuthology.orchestra.run.smithi117.stdout:Total 471 kB/s | 91 kB 00:00 2024-09-15T12:21:09.905 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:21:09.914 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:21:09.914 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:21:09.977 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:21:09.978 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:21:10.128 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:21:10.192 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:22 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:10.269 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:10.281 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:10.325 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:10.326 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:10.326 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:10.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install librbd1 2024-09-15T12:21:10.660 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:10.908 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:10.908 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:10.908 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:21:10.908 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:10.908 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:10.909 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:10.959 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:22 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:11.048 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:11.091 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:11.093 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:11.093 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:11.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rados 2024-09-15T12:21:11.231 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-mirror 2024-09-15T12:21:11.727 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:23 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:11.810 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:24 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:11.816 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:11.859 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:11.860 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:11.860 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:11.927 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rgw 2024-09-15T12:21:11.947 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repo Size 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:21:11.948 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:11.949 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:21:11.949 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:11.949 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 3.5 M 2024-09-15T12:21:11.949 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 30 M 2024-09-15T12:21:11.949 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:21:12.357 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-mirror-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 8.6 MB/s | 3.5 MB 00:00 2024-09-15T12:21:12.358 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:12.358 INFO:teuthology.orchestra.run.smithi117.stdout:Total 8.6 MB/s | 3.5 MB 00:00 2024-09-15T12:21:12.358 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:21:12.369 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:21:12.369 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:21:12.453 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:21:12.453 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:21:12.490 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:24 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:12.580 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:12.624 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:12.625 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:12.625 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:12.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-cephfs 2024-09-15T12:21:13.054 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:21:13.108 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:13.132 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:13.133 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:21:13.133 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T12:21:13.133 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T12:21:13.133 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T12:21:13.133 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:13.256 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:25 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:13.346 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:13.388 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:13.389 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:13.389 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:13.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install python3-rbd 2024-09-15T12:21:14.022 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:26 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:14.112 INFO:teuthology.orchestra.run.smithi103.stdout:Package python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-15T12:21:14.156 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:14.157 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-15T12:21:14.157 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:14.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-fuse 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:14.316 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:14.579 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install rbd-nbd 2024-09-15T12:21:14.805 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:26 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:14.938 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 91 k 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:14.939 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:14.940 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:14.940 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 91 k 2024-09-15T12:21:14.940 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 321 k 2024-09-15T12:21:14.940 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:15.142 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-fuse-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 450 kB/s | 91 kB 00:00 2024-09-15T12:21:15.142 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:15.142 INFO:teuthology.orchestra.run.smithi103.stdout:Total 447 kB/s | 91 kB 00:00 2024-09-15T12:21:15.143 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:15.151 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:15.151 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:15.159 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:05:28 ago on Sun 15 Sep 2024 12:15:47 PM UTC. 2024-09-15T12:21:15.215 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:15.216 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:15.296 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 177 k 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-15T12:21:15.297 INFO:teuthology.orchestra.run.smithi117.stdout:Install 1 Package 2024-09-15T12:21:15.298 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:15.298 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 177 k 2024-09-15T12:21:15.298 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 685 k 2024-09-15T12:21:15.298 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-15T12:21:15.358 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:15.489 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:15.502 INFO:teuthology.orchestra.run.smithi117.stdout:rbd-nbd-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 865 kB/s | 177 kB 00:00 2024-09-15T12:21:15.503 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:15.503 INFO:teuthology.orchestra.run.smithi117.stdout:Total 859 kB/s | 177 kB 00:00 2024-09-15T12:21:15.503 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-15T12:21:15.513 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-15T12:21:15.513 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-15T12:21:15.585 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-15T12:21:15.586 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-15T12:21:15.749 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-15T12:21:15.842 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:15.888 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:16.141 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:16.142 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:16.142 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:16.142 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:16.142 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:16.142 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:16.259 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:16.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-mirror 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:16.533 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-15T12:21:16.806 DEBUG:teuthology.parallel:result is None 2024-09-15T12:21:17.007 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:29 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repo Size 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:17.141 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 3.5 M 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 30 M 2024-09-15T12:21:17.142 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:17.574 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-mirror-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 8.1 MB/s | 3.5 MB 00:00 2024-09-15T12:21:17.575 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:17.576 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.1 MB/s | 3.5 MB 00:00 2024-09-15T12:21:17.576 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:17.587 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:17.587 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:17.671 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:17.671 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:18.267 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:18.333 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:18.359 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:18.359 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T12:21:18.359 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T12:21:18.360 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T12:21:18.360 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T12:21:18.360 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:19.605 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:19.884 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install rbd-nbd 2024-09-15T12:21:20.464 INFO:teuthology.orchestra.run.smithi103.stdout:Last metadata expiration check: 0:05:32 ago on Sun 15 Sep 2024 12:15:48 PM UTC. 2024-09-15T12:21:20.596 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout: Package Architecture Version Repository Size 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 177 k 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:================================================================================ 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout:Install 1 Package 2024-09-15T12:21:20.597 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:20.598 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 177 k 2024-09-15T12:21:20.598 INFO:teuthology.orchestra.run.smithi103.stdout:Installed size: 685 k 2024-09-15T12:21:20.598 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-15T12:21:20.811 INFO:teuthology.orchestra.run.smithi103.stdout:rbd-nbd-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 832 kB/s | 177 kB 00:00 2024-09-15T12:21:20.811 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-15T12:21:20.812 INFO:teuthology.orchestra.run.smithi103.stdout:Total 826 kB/s | 177 kB 00:00 2024-09-15T12:21:20.812 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-15T12:21:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-15T12:21:20.821 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-15T12:21:20.890 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-15T12:21:20.891 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-15T12:21:21.062 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-15T12:21:21.150 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:21.552 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:21.837 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-15T12:21:22.114 DEBUG:teuthology.parallel:result is None 2024-09-15T12:21:22.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-15T12:21:22.263 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T12:21:22.285 INFO:teuthology.orchestra.run.smithi103.stdout:19.3.0-4879.g81fedcb3.el9 2024-09-15T12:21:22.285 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4879.g81fedcb3.el9 2024-09-15T12:21:22.286 INFO:teuthology.task.install:The correct ceph version 19.3.0-4879.g81fedcb3 is installed. 2024-09-15T12:21:22.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-15T12:21:22.425 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T12:21:22.450 INFO:teuthology.orchestra.run.smithi117.stdout:19.3.0-4879.g81fedcb3.el9 2024-09-15T12:21:22.450 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4879.g81fedcb3.el9 2024-09-15T12:21:22.450 INFO:teuthology.task.install:The correct ceph version 19.3.0-4879.g81fedcb3 is installed. 2024-09-15T12:21:22.452 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T12:21:22.452 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:22.452 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T12:21:22.477 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:22.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T12:21:22.519 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T12:21:22.519 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:22.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T12:21:22.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T12:21:22.607 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:22.607 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T12:21:22.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T12:21:22.702 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T12:21:22.703 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:22.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T12:21:22.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T12:21:22.792 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:22.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T12:21:22.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T12:21:22.885 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T12:21:22.886 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:22.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T12:21:22.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T12:21:22.975 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:22.976 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T12:21:23.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T12:21:23.068 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T12:21:23.145 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T12:21:23.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /var/log/ceph 2024-09-15T12:21:23.148 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2024-09-15T12:21:23.173 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T12:21:23.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T12:21:23.213 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T12:21:23.238 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T12:21:23.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T12:21:23.278 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T12:21:23.313 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T12:21:23.313 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T12:21:23.313 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511', 'branch': 'wip-matanb-crimson-testing-59543', 'description': 'crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench}', 'email': None, 'first_in_suite': False, 'job_id': '7905511', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-testing-59543'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 1236, 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'targets': {'smithi103.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDoSiKL10vI0/am/msJYY1PVSH3B5brgaBONgF9qcp2VzYtLoABZ41ns9YCqDFzoyxlada76+GylPlNsWR8/CtU=', 'smithi117.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPSU+2XUwIEggfuXUdK97f5xygMPd/y8YqscMarSGz8LR1Gw91GjB6iiZhcreUYuazejW9+xoV946ticcwmrMVU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'full_sequential': [{'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}, {'radosbench': {'clients': ['client.0'], 'time': 90}}]}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_11:59:21', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T12:21:23.314 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T12:21:23.343 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T12:21:23.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T12:21:23.398 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T12:21:23.423 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:23.423 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T12:21:23.476 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T12:21:23.477 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-15T12:21:23.532 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T12:21:23.532 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:23.532 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-09-15T12:21:23.532 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:23.532 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:23.533 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-15 12:21:18.630614608 +0000 2024-09-15T12:21:23.533 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-15 12:15:07.736333143 +0000 2024-09-15T12:21:23.533 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-15 12:15:07.736333143 +0000 2024-09-15T12:21:23.533 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-15 12:15:07.736333143 +0000 2024-09-15T12:21:23.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T12:21:23.595 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-15T12:21:23.595 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-15T12:21:23.595 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000200885 s, 2.5 MB/s 2024-09-15T12:21:23.597 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T12:21:23.653 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-15T12:21:23.708 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T12:21:23.708 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-15 12:21:18.630614608 +0000 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-15 12:15:08.071326590 +0000 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-15 12:15:08.071326590 +0000 2024-09-15T12:21:23.709 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-15 12:15:08.071326590 +0000 2024-09-15T12:21:23.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T12:21:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-15T12:21:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-15T12:21:23.772 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000229199 s, 2.2 MB/s 2024-09-15T12:21:23.774 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T12:21:23.829 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-15 12:21:18.631614588 +0000 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-15 12:15:08.402320116 +0000 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-15 12:15:08.402320116 +0000 2024-09-15T12:21:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-15 12:15:08.402320116 +0000 2024-09-15T12:21:23.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T12:21:23.949 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-15T12:21:23.949 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-15T12:21:23.949 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000191665 s, 2.7 MB/s 2024-09-15T12:21:23.950 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T12:21:24.007 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-15 12:21:18.631614588 +0000 2024-09-15T12:21:24.062 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-15 12:15:08.745313406 +0000 2024-09-15T12:21:24.063 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-15 12:15:08.745313406 +0000 2024-09-15T12:21:24.063 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-15 12:15:08.745313406 +0000 2024-09-15T12:21:24.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T12:21:24.125 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-15T12:21:24.125 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-15T12:21:24.125 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000211799 s, 2.4 MB/s 2024-09-15T12:21:24.126 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T12:21:24.182 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:24.182 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:24.182 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T12:21:24.201 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T12:21:24.201 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-15T12:21:24.259 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 12:21:13.372871014 +0000 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 12:15:07.726242389 +0000 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 12:15:07.726242389 +0000 2024-09-15T12:21:24.260 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 12:15:07.726242389 +0000 2024-09-15T12:21:24.260 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T12:21:24.326 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T12:21:24.326 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T12:21:24.326 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000219102 s, 2.3 MB/s 2024-09-15T12:21:24.327 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T12:21:24.385 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 12:21:13.373870994 +0000 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 12:15:08.067235672 +0000 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 12:15:08.067235672 +0000 2024-09-15T12:21:24.442 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 12:15:08.067235672 +0000 2024-09-15T12:21:24.443 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T12:21:24.507 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T12:21:24.507 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T12:21:24.507 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00020118 s, 2.5 MB/s 2024-09-15T12:21:24.508 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T12:21:24.565 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:24.621 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 12:21:13.373870994 +0000 2024-09-15T12:21:24.622 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 12:15:08.404229034 +0000 2024-09-15T12:21:24.622 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 12:15:08.404229034 +0000 2024-09-15T12:21:24.622 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 12:15:08.404229034 +0000 2024-09-15T12:21:24.622 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T12:21:24.685 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T12:21:24.685 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T12:21:24.685 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000169568 s, 3.0 MB/s 2024-09-15T12:21:24.686 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T12:21:24.742 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-15T12:21:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T12:21:24.797 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T12:21:24.797 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T12:21:24.797 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T12:21:24.798 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T12:21:24.798 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-15 12:21:13.373870994 +0000 2024-09-15T12:21:24.798 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-15 12:15:08.751222198 +0000 2024-09-15T12:21:24.798 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-15 12:15:08.751222198 +0000 2024-09-15T12:21:24.798 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-15 12:15:08.751222198 +0000 2024-09-15T12:21:24.798 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T12:21:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-15T12:21:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-15T12:21:24.860 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000170236 s, 3.0 MB/s 2024-09-15T12:21:24.862 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T12:21:24.918 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:24.918 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-15T12:21:24.918 INFO:tasks.ceph:Generating config... 2024-09-15T12:21:24.919 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-15T12:21:24.919 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T12:21:24.919 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] bluestore rocksdb cf = False 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] err to stderr = True 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] log flush on exit = True 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] log to file = False 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] log to stderr = True 2024-09-15T12:21:24.920 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd delete sleep = 1 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-15T12:21:24.921 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2024-09-15T12:21:24.921 INFO:tasks.ceph:[client.0] debug ms = 1 2024-09-15T12:21:24.922 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-09-15T12:21:24.922 INFO:tasks.ceph:[client.0] debug rados = 20 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-09-15T12:21:24.922 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2024-09-15T12:21:24.922 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T12:21:24.922 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T12:21:24.922 INFO:tasks.ceph:[mgr] mgr stats period = 30 2024-09-15T12:21:24.922 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-15T12:21:24.923 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-15T12:21:24.923 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-2 2024-09-15T12:21:24.923 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 3-5 2024-09-15T12:21:24.923 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 0-2 2024-09-15T12:21:24.923 INFO:tasks.ceph:[osd.3] crimson seastar cpu cores = 3-5 2024-09-15T12:21:24.923 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T12:21:24.924 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T12:21:24.986 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T12:21:24.989 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T12:21:25.042 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T12:21:25.066 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.103')] 2024-09-15T12:21:25.066 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.103', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'mgr stats period': 30}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}} 2024-09-15T12:21:25.067 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:25.067 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T12:21:25.122 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.103 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:25.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:25.213+0000 7f94ed832f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:25.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:25.213+0000 7f94ed832f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:25.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:25.213+0000 7f94ed832f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: generated fsid 0089f1fe-9ae6-4cff-ad25-a1642089e108 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 0 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 0089f1fe-9ae6-4cff-ad25-a1642089e108 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:last_changed 2024-09-15T12:21:25.214152+0000 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-15T12:21:25.214152+0000 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:min_mon_release 17 (quincy) 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:election_strategy: 1 2024-09-15T12:21:25.215 INFO:teuthology.orchestra.run.smithi103.stdout:0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-15T12:21:25.216 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-15T12:21:25.217 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T12:21:25.231 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0089f1fe-9ae6-4cff-ad25-a1642089e108... 2024-09-15T12:21:25.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T12:21:25.274 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: chdir = "" 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: auth supported = cephx 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: filestore xattr use omap = true 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.314 INFO:teuthology.orchestra.run.smithi103.stdout: mon clock drift allowed = 1.000 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: osd crush chooseleaf type = 0 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: auth debug = true 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on old message = true 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on bug = true 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: mon pg warn max object skew = 0 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default size = 2 2024-09-15T12:21:25.315 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow primary affinity = true 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow pg remap = true 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on legacy crush tunables = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on crush straw calc version zero = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on no sortbitwise = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on osd down out interval zero = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on too few osds = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: mon_allow_pool_size_one = true 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.316 INFO:teuthology.orchestra.run.smithi103.stdout: osd default data pool replay window = 5 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: mon allow pool delete = true 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: mon cluster log file level = debug 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: debug asserts on shutdown = true 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: mon health detail to clog = false 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: mon host = 172.21.15.103 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms client mode = crc 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms cluster mode = crc 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms mon client mode = crc 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms mon cluster mode = crc 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms mon service mode = crc 2024-09-15T12:21:25.317 INFO:teuthology.orchestra.run.smithi103.stdout: ms service mode = crc 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_min_size = 2 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_size = 3 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 0089f1fe-9ae6-4cff-ad25-a1642089e108 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout:[osd] 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd journal size = 100 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub load threshold = 5.0 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub max interval = 120 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd mclock profile = high_recovery_ops 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd recover clone overlap = true 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: osd recovery max chunk = 1048576 2024-09-15T12:21:25.318 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug shutdown = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug op order = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify stray on activate = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug trim objects = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd open classes on start = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug pg log writeout = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd deep scrub update digest min age = 30 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: osd map max advance = 10 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: journal zero on create = true 2024-09-15T12:21:25.319 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: filestore ondisk finisher threads = 3 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: filestore apply finisher threads = 3 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bdev debug aio = true 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug misdirected ops = true 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: debug monc = 20 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore block size = 96636764160 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore compression algorithm = snappy 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore compression mode = aggressive 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore fsck on mount = True 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore rocksdb cf = False 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: crimson alien thread cpu cores = 6-7 2024-09-15T12:21:25.320 INFO:teuthology.orchestra.run.smithi103.stdout: crimson osd obc lru size = 10 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluefs = 20 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluestore = 20 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: debug osd = 20 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: debug rocksdb = 10 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: err to stderr = True 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: log flush on exit = True 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: log to file = False 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: log to stderr = True 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd full ratio = 0.9 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd nearfull ratio = 0.8 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: osd blocked scrub grace period = 3600 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug reject backfill probability = 0.3 2024-09-15T12:21:25.321 INFO:teuthology.orchestra.run.smithi103.stdout: osd delete sleep = 1 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd failsafe full ratio = 0.95 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd max backfills = 3 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd max markdown count = 1000 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd objectstore = bluestore 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub min interval = 60 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd shutdown pgref assert = True 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: osd snap trim sleep = 2 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr] 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: debug mgr = 20 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-09-15T12:21:25.322 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mgr/telemetry/nag = false 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mgr stats period = 30 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout:[mon] 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: debug paxos = 20 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon data avail warn = 5 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon mgr mkfs grace = 240 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd reporter subtree level = osd 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd prime pg temp = true 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.323 INFO:teuthology.orchestra.run.smithi103.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: # 1m isn't quite enough 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon_down_mkfs_grace = 2m 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_filestore_osds = false 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon min osdmap epochs = 50 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune interval = 2 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune min = 15 2024-09-15T12:21:25.324 INFO:teuthology.orchestra.run.smithi103.stdout: mon osdmap full prune txsize = 2 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default crimson = True 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: paxos service trim min = 10 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout:[client] 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: rgw cache enabled = true 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable ops log = true 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable usage log = true 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.a] 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: debug objecter = 20 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout: debug rados = 20 2024-09-15T12:21:25.325 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.0] 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout: crimson seastar cpu cores = 0-2 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.1] 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout: crimson seastar cpu cores = 3-5 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.2] 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout: crimson seastar cpu cores = 0-2 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.3] 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi103.stdout: crimson seastar cpu cores = 3-5 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2024-09-15T12:21:25.326 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.327 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2024-09-15T12:21:25.328 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2024-09-15T12:21:25.329 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = 172.21.15.103 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms client mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms cluster mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms mon client mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms mon cluster mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms mon service mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: ms service mode = crc 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_min_size = 2 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_size = 3 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = 0089f1fe-9ae6-4cff-ad25-a1642089e108 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.330 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 120 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd mclock profile = high_recovery_ops 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug trim objects = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.331 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: debug monc = 20 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore compression algorithm = snappy 2024-09-15T12:21:25.332 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore compression mode = aggressive 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore rocksdb cf = False 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: crimson alien thread cpu cores = 6-7 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: crimson osd obc lru size = 10 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: err to stderr = True 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: log flush on exit = True 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: log to file = False 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: log to stderr = True 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2024-09-15T12:21:25.333 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd blocked scrub grace period = 3600 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug reject backfill probability = 0.3 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd delete sleep = 1 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd max backfills = 3 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd max markdown count = 1000 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub min interval = 60 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd shutdown pgref assert = True 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: osd snap trim sleep = 2 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-09-15T12:21:25.334 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mgr stats period = 30 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2024-09-15T12:21:25.335 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon min osdmap epochs = 50 2024-09-15T12:21:25.336 INFO:teuthology.orchestra.run.smithi117.stdout: mon osdmap full prune interval = 2 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: mon osdmap full prune min = 15 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: mon osdmap full prune txsize = 2 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default crimson = True 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: paxos service trim min = 10 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-09-15T12:21:25.337 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: debug objecter = 20 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: debug rados = 20 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: crimson seastar cpu cores = 0-2 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: crimson seastar cpu cores = 3-5 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: crimson seastar cpu cores = 0-2 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2024-09-15T12:21:25.338 INFO:teuthology.orchestra.run.smithi117.stdout: crimson seastar cpu cores = 3-5 2024-09-15T12:21:25.339 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T12:21:25.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T12:21:25.444 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T12:21:25.445 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:25.445 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T12:21:25.460 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:25.460 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T12:21:25.515 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:21:25.515 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:25.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T12:21:25.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T12:21:25.589 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:25.589 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:25.644 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:21:25.644 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:25.644 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T12:21:25.645 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T12:21:25.680 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:25.680 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:25.736 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T12:21:25.736 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T12:21:25.736 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T12:21:25.844 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T12:21:25.847 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T12:21:25.847 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T12:21:25.847 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T12:21:25.897 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T12:21:25.910 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-15T12:21:25.959 INFO:teuthology.orchestra.run.smithi117.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-15T12:21:25.972 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T12:21:25.972 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2024-09-15T12:21:25.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T12:21:25.997 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:25.997 INFO:tasks.ceph:role: osd.0 2024-09-15T12:21:25.997 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:21:25.997 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T12:21:26.068 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T12:21:26.068 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T12:21:26.068 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T12:21:26.068 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T12:21:26.069 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T12:21:26.579 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-09-15T12:21:26.582 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-09-15T12:21:26.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T12:21:26.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T12:21:26.765 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T12:21:26.831 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:26.831 INFO:tasks.ceph:role: osd.1 2024-09-15T12:21:26.832 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:21:26.832 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T12:21:26.904 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-09-15T12:21:26.905 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T12:21:26.905 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T12:21:26.905 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T12:21:26.905 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T12:21:27.351 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-09-15T12:21:27.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-09-15T12:21:27.353 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T12:21:27.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T12:21:27.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:27.781 INFO:teuthology.orchestra.run.smithi103.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:27.782 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:27,781 seastar - Reactor backend: linux-aio 2024-09-15T12:21:27.796 INFO:teuthology.orchestra.run.smithi103.stderr:==27753==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:27.919 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:27,919 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-15T12:21:27.919 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:27,919 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-15T12:21:27.920 INFO:teuthology.orchestra.run.smithi103.stderr:DEBUG 2024-09-15 12:21:27,920 [shard 0:main] seastar - reactor::drain 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr:Suppressions used: 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr: count bytes template 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:28.449 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:21:28.508 INFO:teuthology.orchestra.run.smithi103.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:28.512 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:28,512 seastar - Reactor backend: linux-aio 2024-09-15T12:21:28.522 INFO:teuthology.orchestra.run.smithi103.stderr:==27751==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:28.587 INFO:teuthology.orchestra.run.smithi103.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x133c2d 0xdd332 0x4ab1c 0x3613f13 0x5550a31 0x5550e5b 0x5fb4987 0x5fb8f6a 0x5fb946c 0x5e00fef 0x5e19231 0x71579d2 0x71691a5 0x716a642 0x716a6c7 0x716a7c9 0x716a8c1 0x716aace 0x716acb4 0x716b003 0x716b3a7 0x716b5d7 0x716b841 0x716bb0e 0x716bd2c 0x716bde4 0x716beaa 0x716bf8b 0x5c63e30 0x5c63f3a 0x5c63ff6 0x5c640b2 0x5c6416e 0x5c642bb 0x5c64446 0x5c64807 0x5c7e60a 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-15T12:21:28.587 INFO:teuthology.orchestra.run.smithi103.stderr:kernel callstack: 2024-09-15T12:21:33.216 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:33.217 INFO:teuthology.orchestra.run.smithi103.stderr:Suppressions used: 2024-09-15T12:21:33.217 INFO:teuthology.orchestra.run.smithi103.stderr: count bytes template 2024-09-15T12:21:33.217 INFO:teuthology.orchestra.run.smithi103.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:33.217 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:33.217 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:21:33.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T12:21:33.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:33.552 INFO:teuthology.orchestra.run.smithi103.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:33.553 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:33,552 seastar - Reactor backend: linux-aio 2024-09-15T12:21:33.558 INFO:teuthology.orchestra.run.smithi103.stderr:==27940==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:33.681 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:33,680 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-15T12:21:33.681 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:33,680 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-15T12:21:33.681 INFO:teuthology.orchestra.run.smithi103.stderr:DEBUG 2024-09-15 12:21:33,681 [shard 0:main] seastar - reactor::drain 2024-09-15T12:21:34.183 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:34.183 INFO:teuthology.orchestra.run.smithi103.stderr:Suppressions used: 2024-09-15T12:21:34.184 INFO:teuthology.orchestra.run.smithi103.stderr: count bytes template 2024-09-15T12:21:34.184 INFO:teuthology.orchestra.run.smithi103.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:34.184 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:34.184 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:21:34.251 INFO:teuthology.orchestra.run.smithi103.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:34.255 INFO:teuthology.orchestra.run.smithi103.stderr:INFO 2024-09-15 12:21:34,254 seastar - Reactor backend: linux-aio 2024-09-15T12:21:34.274 INFO:teuthology.orchestra.run.smithi103.stderr:==27938==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr:Suppressions used: 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr: count bytes template 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr:----------------------------------------------------- 2024-09-15T12:21:38.962 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:21:38.983 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T12:21:39.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T12:21:39.038 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:39.038 INFO:tasks.ceph:role: osd.2 2024-09-15T12:21:39.038 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:21:39.038 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T12:21:39.115 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T12:21:39.115 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T12:21:39.115 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T12:21:39.115 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T12:21:39.116 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T12:21:39.618 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-15T12:21:39.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-15T12:21:39.621 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-15T12:21:39.763 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T12:21:39.795 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T12:21:39.860 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2024-09-15T12:21:39.860 INFO:tasks.ceph:role: osd.3 2024-09-15T12:21:39.860 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:21:39.860 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2024-09-15T12:21:39.946 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T12:21:39.947 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T12:21:39.947 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T12:21:39.947 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T12:21:40.391 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2024-09-15T12:21:40.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2024-09-15T12:21:40.394 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2024-09-15T12:21:40.436 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T12:21:40.506 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:40.799 INFO:teuthology.orchestra.run.smithi117.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:40.800 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:40,800 seastar - Reactor backend: linux-aio 2024-09-15T12:21:40.815 INFO:teuthology.orchestra.run.smithi117.stderr:==28055==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:40,936 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-15T12:21:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:40,936 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-2 2024-09-15T12:21:40.937 INFO:teuthology.orchestra.run.smithi117.stderr:DEBUG 2024-09-15 12:21:40,937 [shard 0:main] seastar - reactor::drain 2024-09-15T12:21:41.456 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:41.456 INFO:teuthology.orchestra.run.smithi117.stderr:Suppressions used: 2024-09-15T12:21:41.456 INFO:teuthology.orchestra.run.smithi117.stderr: count bytes template 2024-09-15T12:21:41.457 INFO:teuthology.orchestra.run.smithi117.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:41.457 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:41.457 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T12:21:41.514 INFO:teuthology.orchestra.run.smithi117.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:41.518 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:41,517 seastar - Reactor backend: linux-aio 2024-09-15T12:21:41.527 INFO:teuthology.orchestra.run.smithi117.stderr:==28052==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr:Suppressions used: 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr: count bytes template 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:46.204 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T12:21:46.225 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T12:21:46.250 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:46.522 INFO:teuthology.orchestra.run.smithi117.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:46.523 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:46,523 seastar - Reactor backend: linux-aio 2024-09-15T12:21:46.529 INFO:teuthology.orchestra.run.smithi117.stderr:==28242==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:46.651 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:46,651 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-15T12:21:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:46,651 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 3-5 2024-09-15T12:21:46.652 INFO:teuthology.orchestra.run.smithi117.stderr:DEBUG 2024-09-15 12:21:46,652 [shard 0:main] seastar - reactor::drain 2024-09-15T12:21:47.154 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:47.155 INFO:teuthology.orchestra.run.smithi117.stderr:Suppressions used: 2024-09-15T12:21:47.155 INFO:teuthology.orchestra.run.smithi117.stderr: count bytes template 2024-09-15T12:21:47.155 INFO:teuthology.orchestra.run.smithi117.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:47.155 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:47.155 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T12:21:47.208 INFO:teuthology.orchestra.run.smithi117.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-15T12:21:47.212 INFO:teuthology.orchestra.run.smithi117.stderr:INFO 2024-09-15 12:21:47,211 seastar - Reactor backend: linux-aio 2024-09-15T12:21:47.229 INFO:teuthology.orchestra.run.smithi117.stderr:==28240==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-15T12:21:52.401 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:52.401 INFO:teuthology.orchestra.run.smithi117.stderr:Suppressions used: 2024-09-15T12:21:52.402 INFO:teuthology.orchestra.run.smithi117.stderr: count bytes template 2024-09-15T12:21:52.402 INFO:teuthology.orchestra.run.smithi117.stderr: 1 8 MallocExtension::Register 2024-09-15T12:21:52.402 INFO:teuthology.orchestra.run.smithi117.stderr:----------------------------------------------------- 2024-09-15T12:21:52.402 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T12:21:52.423 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T12:21:52.447 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T12:21:52.448 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:52.448 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T12:21:52.478 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:52.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T12:21:52.546 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:52.546 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T12:21:52.570 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:52.570 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T12:21:52.634 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:52.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T12:21:52.699 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:52.699 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T12:21:52.716 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:52.716 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-15T12:21:52.755 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T12:21:52.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T12:21:52.782 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.0] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBI0eZmcpT6JxAAYhH2nZoUfB7KrTrmoIdChg== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.1] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBO0eZmGtgOGRAAR/JwdDNGv+jqbqmIGdjs/A== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.x] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBF0eZmk7tJMhAALkocHPp3GPtREkMe3RJq4w== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.2] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBV0eZmMpMfKBAArfWN/pPtXzXVXTD+91XRfg== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.3] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBb0eZm7k9mFhAAOd1fNri59W8870pfXBbfGQ== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBF0eZmG1xzNRAA4ifRF6eohMf4LfaXWRvJWA== 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout:[client.1] 2024-09-15T12:21:52.783 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQBF0eZmEH0sORAA93RCtocRSEgWTdEGqA1Brw== 2024-09-15T12:21:52.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T12:21:52.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T12:21:52.952 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T12:21:53.012 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T12:21:53.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T12:21:53.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T12:21:53.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T12:21:53.262 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T12:21:53.262 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T12:21:53.287 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T12:21:53.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.397+0000 7fa7a6030c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.397+0000 7fa7a6030c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.413 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T12:21:53.437 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T12:21:53.492 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T12:21:53.492 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T12:21:53.492 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T12:21:53.534 INFO:tasks.ceph.mon.a:Started 2024-09-15T12:21:53.534 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T12:21:53.535 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T12:21:53.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T12:21:53.537 INFO:tasks.ceph.mgr.x:Started 2024-09-15T12:21:53.538 DEBUG:tasks.ceph:set 0 configs 2024-09-15T12:21:53.538 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T12:21:53.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T12:21:53.616 INFO:tasks.ceph.mon.a.smithi103.stderr:2024-09-15T12:21:53.613+0000 7f9d316d7c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.616 INFO:tasks.ceph.mon.a.smithi103.stderr:2024-09-15T12:21:53.613+0000 7f9d316d7c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.616 INFO:tasks.ceph.mon.a.smithi103.stderr:2024-09-15T12:21:53.614+0000 7f9d316d7c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.642 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-15T12:21:53.641+0000 7f80fe3c7000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.637+0000 7fc468d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.649 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-15T12:21:53.648+0000 7f80fe3c7000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.650 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-15T12:21:53.648+0000 7f80fe3c7000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.648+0000 7fc468d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.719 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-15T12:21:53.731 INFO:tasks.ceph:check_enable_crimson: True 2024-09-15T12:21:53.731 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-15T12:21:53.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-15T12:21:53.836 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.829+0000 7fa077791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:53.838+0000 7fa077791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr:2024-09-15T12:21:53.915+0000 7f9d283f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr:Please be aware that this feature is experimental, untested, 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr:unsupported, and may result in data corruption, data loss, 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr:feature with important data. 2024-09-15T12:21:53.917 INFO:tasks.ceph.mon.a.smithi103.stderr: 2024-09-15T12:21:54.736 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T12:21:54.736 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:54.737 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T12:21:54.764 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-15T12:21:54.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T12:21:54.830 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:54.830 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T12:21:54.856 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-15T12:21:54.856 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T12:21:54.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 37772177-3b71-4af7-b52e-b013e757f215 0 2024-09-15T12:21:55.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.060+0000 7f0793591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.070+0000 7f0793591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.150 INFO:teuthology.orchestra.run.smithi117.stdout:0 2024-09-15T12:21:55.162 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 5a23b5b3-b2de-48d3-b668-2ebed6c50787 1 2024-09-15T12:21:55.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.262+0000 7f02af191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.271+0000 7f02af191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.351 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-15T12:21:55.363 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 97c17cfc-5763-433b-b175-3e1ba3a87f8a 2 2024-09-15T12:21:55.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.463+0000 7fcded191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.472+0000 7fcded191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.552 INFO:teuthology.orchestra.run.smithi117.stdout:2 2024-09-15T12:21:55.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 502045c7-d23c-4241-9fbd-971c75c81272 3 2024-09-15T12:21:55.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.664+0000 7f2275f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-15T12:21:55.673+0000 7f2275f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:55.756 INFO:teuthology.orchestra.run.smithi117.stdout:3 2024-09-15T12:21:55.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-15T12:21:55.802 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T12:21:55.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-15T12:21:55.844 INFO:tasks.ceph.osd.0:Started 2024-09-15T12:21:55.844 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-15T12:21:55.875 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:21:55.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:21:55.918 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:21:55.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-15T12:21:55.948 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T12:21:55.948 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-15T12:21:55.990 INFO:tasks.ceph.osd.2:Started 2024-09-15T12:21:55.990 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2024-09-15T12:21:56.019 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:21:56.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:21:56.062 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:21:56.062 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:21:56.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:56.170+0000 7f104df91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:56.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:21:56.178+0000 7f104df91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:56.261 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:21:56.261 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":7,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:21:55.751033+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:21:56.272 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T12:21:56.273 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T12:21:56.676 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T12:21:56.780 INFO:teuthology.misc.health.smithi103.stderr:2024-09-15T12:21:56.773+0000 7fe13a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:56.784 INFO:teuthology.misc.health.smithi103.stderr:2024-09-15T12:21:56.782+0000 7fe13a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:21:56.850 INFO:teuthology.misc.health.smithi103.stdout: 2024-09-15T12:21:56.850 INFO:teuthology.misc.health.smithi103.stdout:{"epoch":7,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:21:55.751033+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:21:56.861 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-15T12:21:59.003 INFO:tasks.ceph.mgr.x.smithi117.stderr:2024-09-15T12:21:59.001+0000 7f809b59e640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-15T12:22:03.265 DEBUG:teuthology.orchestra.run.smithi103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T12:22:03.369 INFO:teuthology.misc.health.smithi103.stderr:2024-09-15T12:22:03.363+0000 7f770f991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:03.372 INFO:teuthology.misc.health.smithi103.stderr:2024-09-15T12:22:03.370+0000 7f770f991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:03.571 INFO:teuthology.misc.health.smithi103.stdout: 2024-09-15T12:22:03.571 INFO:teuthology.misc.health.smithi103.stdout:{"epoch":11,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:22:03.023545+0000","last_up_change":"2024-09-15T12:22:02.016608+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4067805257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4067805257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4067805257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4067805257}]},"public_addr":"172.21.15.103:6801/4067805257","cluster_addr":"172.21.15.103:6800/4067805257","heartbeat_back_addr":"172.21.15.103:6802/4067805257","heartbeat_front_addr":"172.21.15.103:6805/4067805257","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":1468721707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1468721707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1468721707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":1468721707}]},"public_addr":"172.21.15.103:6803/1468721707","cluster_addr":"172.21.15.103:6804/1468721707","heartbeat_back_addr":"172.21.15.103:6808/1468721707","heartbeat_front_addr":"172.21.15.103:6809/1468721707","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":3270309961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3270309961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":3270309961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3270309961}]},"public_addr":"172.21.15.117:6803/3270309961","cluster_addr":"172.21.15.117:6802/3270309961","heartbeat_back_addr":"172.21.15.117:6817/3270309961","heartbeat_front_addr":"172.21.15.117:6816/3270309961","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":810119942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":810119942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":810119942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":810119942}]},"public_addr":"172.21.15.117:6805/810119942","cluster_addr":"172.21.15.117:6806/810119942","heartbeat_back_addr":"172.21.15.117:6813/810119942","heartbeat_front_addr":"172.21.15.117:6814/810119942","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:22:03.583 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-15T12:22:03.583 INFO:tasks.ceph:Creating RBD pool 2024-09-15T12:22:03.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T12:22:03.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:03.690+0000 7f9d220a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:03.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:03.702+0000 7f9d220a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'rbd' created 2024-09-15T12:22:04.054 DEBUG:teuthology.orchestra.run.smithi103:> rbd --cluster ceph pool init rbd 2024-09-15T12:22:04.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:04.165+0000 7f1f44e44ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:04.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:04.165+0000 7f1f44e44ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:04.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:04.170+0000 7f1f44e44ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.088 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T12:22:07.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T12:22:07.089 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T12:22:07.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.193+0000 7f492ab91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.202+0000 7f492ab91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.389 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:07.402 INFO:teuthology.orchestra.run.smithi103.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2024-09-15T12:22:07.402 INFO:tasks.ceph_manager:config epoch is 1 2024-09-15T12:22:07.403 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T12:22:07.403 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T12:22:07.403 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T12:22:07.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.505+0000 7f285c191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.513+0000 7f285c191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.750 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:07.762 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":1809814645},{"type":"v1","addr":"172.21.15.117:6801","nonce":1809814645}]},"active_addr":"172.21.15.117:6801/1809814645","active_change":"2024-09-15T12:21:57.992715+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1650672491}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":488321181}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":577544831}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3214163268}]}]} 2024-09-15T12:22:07.764 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T12:22:07.764 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T12:22:07.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:22:07.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.865+0000 7f7caa391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:07.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:07.873+0000 7f7caa391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.063 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:08.063 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":15,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:22:07.045031+0000","last_up_change":"2024-09-15T12:22:02.016608+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T12:22:03.901604+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4067805257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4067805257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4067805257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4067805257}]},"public_addr":"172.21.15.103:6801/4067805257","cluster_addr":"172.21.15.103:6800/4067805257","heartbeat_back_addr":"172.21.15.103:6802/4067805257","heartbeat_front_addr":"172.21.15.103:6805/4067805257","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":1468721707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1468721707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1468721707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":1468721707}]},"public_addr":"172.21.15.103:6803/1468721707","cluster_addr":"172.21.15.103:6804/1468721707","heartbeat_back_addr":"172.21.15.103:6808/1468721707","heartbeat_front_addr":"172.21.15.103:6809/1468721707","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":3270309961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3270309961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":3270309961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3270309961}]},"public_addr":"172.21.15.117:6803/3270309961","cluster_addr":"172.21.15.117:6802/3270309961","heartbeat_back_addr":"172.21.15.117:6817/3270309961","heartbeat_front_addr":"172.21.15.117:6816/3270309961","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":810119942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":810119942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":810119942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":810119942}]},"public_addr":"172.21.15.117:6805/810119942","cluster_addr":"172.21.15.117:6806/810119942","heartbeat_back_addr":"172.21.15.117:6813/810119942","heartbeat_front_addr":"172.21.15.117:6814/810119942","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:22:08.076 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T12:22:08.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:22:08.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.177+0000 7fd039911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.185+0000 7fd039911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:08.371 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":15,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:22:07.045031+0000","last_up_change":"2024-09-15T12:22:02.016608+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T12:22:03.901604+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4067805257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4067805257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4067805257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4067805257}]},"public_addr":"172.21.15.103:6801/4067805257","cluster_addr":"172.21.15.103:6800/4067805257","heartbeat_back_addr":"172.21.15.103:6802/4067805257","heartbeat_front_addr":"172.21.15.103:6805/4067805257","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":1468721707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":1468721707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1468721707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6809","nonce":1468721707}]},"public_addr":"172.21.15.103:6803/1468721707","cluster_addr":"172.21.15.103:6804/1468721707","heartbeat_back_addr":"172.21.15.103:6808/1468721707","heartbeat_front_addr":"172.21.15.103:6809/1468721707","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":3270309961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3270309961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":3270309961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":3270309961}]},"public_addr":"172.21.15.117:6803/3270309961","cluster_addr":"172.21.15.117:6802/3270309961","heartbeat_back_addr":"172.21.15.117:6817/3270309961","heartbeat_front_addr":"172.21.15.117:6816/3270309961","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6805","nonce":810119942}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":810119942}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6813","nonce":810119942}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":810119942}]},"public_addr":"172.21.15.117:6805/810119942","cluster_addr":"172.21.15.117:6806/810119942","heartbeat_back_addr":"172.21.15.117:6813/810119942","heartbeat_front_addr":"172.21.15.117:6814/810119942","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:22:08.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T12:22:08.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T12:22:08.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T12:22:08.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T12:22:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.497+0000 7f099c791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.497+0000 7f2e4329c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.497+0000 7f70a5991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.497+0000 7f5a000d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.507+0000 7f2e4329c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.507+0000 7f099c791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.507+0000 7f70a5991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.507+0000 7f5a000d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.543 INFO:teuthology.orchestra.run.smithi103.stdout:42949672966 2024-09-15T12:22:08.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T12:22:08.549 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.549 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T12:22:08.550 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T12:22:08.551 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.552 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T12:22:08.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.652+0000 7f81e9391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.663+0000 7f81e9391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.700+0000 7f4457591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.703+0000 7f12bc708640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.710+0000 7f4457591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.713+0000 7f12bc708640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.716 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.707+0000 7f1751151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:08.724+0000 7f1751151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:08.895 INFO:teuthology.orchestra.run.smithi103.stdout:42949672966 2024-09-15T12:22:08.910 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.3 2024-09-15T12:22:08.912 DEBUG:teuthology.parallel:result is None 2024-09-15T12:22:08.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.0 2024-09-15T12:22:08.924 DEBUG:teuthology.parallel:result is None 2024-09-15T12:22:08.926 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.2 2024-09-15T12:22:08.939 DEBUG:teuthology.parallel:result is None 2024-09-15T12:22:08.942 INFO:teuthology.orchestra.run.smithi103.stdout:38654705671 2024-09-15T12:22:08.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.1 2024-09-15T12:22:08.955 DEBUG:teuthology.parallel:result is None 2024-09-15T12:22:08.955 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T12:22:08.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:09.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.100+0000 7f5163b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.110 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.108+0000 7f5163b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.294 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:09.295 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:09.307 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-15T12:22:08.539625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108064,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379296,"statfs":{"total":386547056640,"available":386436399104,"internally_reserved":0,"allocated":704512,"data_stored":1907549,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:07.061503+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062312+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:07.062312+0000","last_peered":"2024-09-15T12:22:07.062312+0000","last_clean":"2024-09-15T12:22:07.062312+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:07.062312+0000","last_undegraded":"2024-09-15T12:22:07.062312+0000","last_fullsized":"2024-09-15T12:22:07.062312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:07.062090+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062168+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:07.062168+0000","last_peered":"2024-09-15T12:22:07.062168+0000","last_clean":"2024-09-15T12:22:07.062168+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:07.062168+0000","last_undegraded":"2024-09-15T12:22:07.062168+0000","last_fullsized":"2024-09-15T12:22:07.062168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063698+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:07.063698+0000","last_peered":"2024-09-15T12:22:07.063698+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:07.063698+0000","last_undegraded":"2024-09-15T12:22:07.063698+0000","last_fullsized":"2024-09-15T12:22:07.063698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:07.058522+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.065018+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:07.065018+0000","last_peered":"2024-09-15T12:22:07.065018+0000","last_clean":"2024-09-15T12:22:07.065018+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:07.065018+0000","last_undegraded":"2024-09-15T12:22:07.065018+0000","last_fullsized":"2024-09-15T12:22:07.065018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:07.063684+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:07.058645+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:09.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:09.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.410+0000 7f7b53151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.418+0000 7f7b53151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.609 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:09.609 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:09.621 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-15T12:22:08.539625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108064,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379296,"statfs":{"total":386547056640,"available":386436399104,"internally_reserved":0,"allocated":704512,"data_stored":1907549,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:07.061503+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062312+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:07.062312+0000","last_peered":"2024-09-15T12:22:07.062312+0000","last_clean":"2024-09-15T12:22:07.062312+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:07.062312+0000","last_undegraded":"2024-09-15T12:22:07.062312+0000","last_fullsized":"2024-09-15T12:22:07.062312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:07.062090+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062168+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:07.062168+0000","last_peered":"2024-09-15T12:22:07.062168+0000","last_clean":"2024-09-15T12:22:07.062168+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:07.062168+0000","last_undegraded":"2024-09-15T12:22:07.062168+0000","last_fullsized":"2024-09-15T12:22:07.062168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063698+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:07.063698+0000","last_peered":"2024-09-15T12:22:07.063698+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:07.063698+0000","last_undegraded":"2024-09-15T12:22:07.063698+0000","last_fullsized":"2024-09-15T12:22:07.063698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:07.058522+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.065018+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:07.065018+0000","last_peered":"2024-09-15T12:22:07.065018+0000","last_clean":"2024-09-15T12:22:07.065018+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:07.065018+0000","last_undegraded":"2024-09-15T12:22:07.065018+0000","last_fullsized":"2024-09-15T12:22:07.065018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:07.063684+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:07.058645+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:09.622 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T12:22:09.622 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T12:22:09.622 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T12:22:09.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T12:22:09.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.725+0000 7fdee6991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:09.731+0000 7fdee6991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:09.953 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:09.953 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T12:22:09.966 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T12:22:09.967 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T12:22:09.980 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T12:22:09.980 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-matanb-crimson-testing-59543'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore rocksdb cf': False, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'err to stderr': True, 'log flush on exit': True, 'log to file': False, 'log to stderr': True, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'osd.0': {'crimson seastar cpu cores': '0-2'}, 'osd.1': {'crimson seastar cpu cores': '3-5'}, 'osd.2': {'crimson seastar cpu cores': '0-2'}, 'osd.3': {'crimson seastar cpu cores': '3-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum'], 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}} 2024-09-15T12:22:09.980 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T12:22:09.980 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T12:22:10.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.085+0000 7fa22cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.097+0000 7fa22cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.300 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T12:22:10.301 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T12:22:10.301 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:10.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.402+0000 7fc0c52d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.410+0000 7fc0c52d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.599 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:10.600 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:10.612 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-15T12:22:10.000573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108064,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379296,"statfs":{"total":386547056640,"available":386436399104,"internally_reserved":0,"allocated":704512,"data_stored":1907549,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.460947"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:07.061503+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062312+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:07.062312+0000","last_peered":"2024-09-15T12:22:07.062312+0000","last_clean":"2024-09-15T12:22:07.062312+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:07.062312+0000","last_undegraded":"2024-09-15T12:22:07.062312+0000","last_fullsized":"2024-09-15T12:22:07.062312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:07.062090+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062168+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:07.062168+0000","last_peered":"2024-09-15T12:22:07.062168+0000","last_clean":"2024-09-15T12:22:07.062168+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:07.062168+0000","last_undegraded":"2024-09-15T12:22:07.062168+0000","last_fullsized":"2024-09-15T12:22:07.062168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063698+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:07.063698+0000","last_peered":"2024-09-15T12:22:07.063698+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:07.063698+0000","last_undegraded":"2024-09-15T12:22:07.063698+0000","last_fullsized":"2024-09-15T12:22:07.063698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:07.058522+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.065018+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:07.065018+0000","last_peered":"2024-09-15T12:22:07.065018+0000","last_clean":"2024-09-15T12:22:07.065018+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:07.065018+0000","last_undegraded":"2024-09-15T12:22:07.065018+0000","last_fullsized":"2024-09-15T12:22:07.065018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:07.063684+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:07.058645+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:10.613 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:10.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.715+0000 7f728c191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:10.723+0000 7f728c191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:10.909 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:10.910 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:10.922 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-15T12:22:10.000573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108064,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379296,"statfs":{"total":386547056640,"available":386436399104,"internally_reserved":0,"allocated":704512,"data_stored":1907549,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.460947"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:07.061503+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062312+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:07.062312+0000","last_peered":"2024-09-15T12:22:07.062312+0000","last_clean":"2024-09-15T12:22:07.062312+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:07.062312+0000","last_undegraded":"2024-09-15T12:22:07.062312+0000","last_fullsized":"2024-09-15T12:22:07.062312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:07.062090+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062168+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:07.062168+0000","last_peered":"2024-09-15T12:22:07.062168+0000","last_clean":"2024-09-15T12:22:07.062168+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:07.062168+0000","last_undegraded":"2024-09-15T12:22:07.062168+0000","last_fullsized":"2024-09-15T12:22:07.062168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063698+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:07.063698+0000","last_peered":"2024-09-15T12:22:07.063698+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:07.063698+0000","last_undegraded":"2024-09-15T12:22:07.063698+0000","last_fullsized":"2024-09-15T12:22:07.063698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:07.058522+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.065018+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:07.065018+0000","last_peered":"2024-09-15T12:22:07.065018+0000","last_clean":"2024-09-15T12:22:07.065018+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:07.065018+0000","last_undegraded":"2024-09-15T12:22:07.065018+0000","last_fullsized":"2024-09-15T12:22:07.065018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:07.063684+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:07.058645+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:10.923 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T12:22:10.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T12:22:11.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.027+0000 7f41676d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.034+0000 7f41676d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.222 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 15 2024-09-15T12:22:11.222 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 0089f1fe-9ae6-4cff-ad25-a1642089e108 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-15T12:21:53.636578+0000 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-15T12:22:07.045031+0000 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 4 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client jewel 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:allow_crimson=true 2024-09-15T12:22:11.223 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 12 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 15 flags hashpspool,nopgchange,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 1.50 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout: removed_snaps_queue [2~1] 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 4 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6801/4067805257 v2:172.21.15.103:6800/4067805257 exists,up 37772177-3b71-4af7-b52e-b013e757f215 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6803/1468721707 v2:172.21.15.103:6804/1468721707 exists,up 5a23b5b3-b2de-48d3-b668-2ebed6c50787 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.117:6803/3270309961 v2:172.21.15.117:6802/3270309961 exists,up 97c17cfc-5763-433b-b175-3e1ba3a87f8a 2024-09-15T12:22:11.224 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.117:6805/810119942 v2:172.21.15.117:6806/810119942 exists,up 502045c7-d23c-4241-9fbd-971c75c81272 2024-09-15T12:22:11.235 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6801/4067805257 v2:172.21.15.103:6800/4067805257 exists,up 37772177-3b71-4af7-b52e-b013e757f215', 'osd.1 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.103:6803/1468721707 v2:172.21.15.103:6804/1468721707 exists,up 5a23b5b3-b2de-48d3-b668-2ebed6c50787', 'osd.2 up in weight 1 up_from 9 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.117:6803/3270309961 v2:172.21.15.117:6802/3270309961 exists,up 97c17cfc-5763-433b-b175-3e1ba3a87f8a', 'osd.3 up in weight 1 up_from 10 up_thru 12 down_at 0 last_clean_interval [0,0) v2:172.21.15.117:6805/810119942 v2:172.21.15.117:6806/810119942 exists,up 502045c7-d23c-4241-9fbd-971c75c81272'] 2024-09-15T12:22:11.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mon.a", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster": "ceph", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "admin_socket_mode": "", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_client_required": "cephx, none", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_cluster_required": "cephx", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_debug": "true", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T12:22:11.343 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_service_required": "cephx", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "auth_supported": "cephx", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_aio": "true", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_aio_reap_max": "16", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_async_discard_threads": "0", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_block_size": "4096", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_debug_aio": "false", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T12:22:11.344 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_enable_discard": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_flock_retry": "3", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_inject_crash": "0", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_ioring": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_ioring_hipri": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T12:22:11.345 INFO:teuthology.orchestra.run.smithi103.stdout: "bdev_type": "", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_allocator": "hybrid", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_buffered_io": "true", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T12:22:11.346 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_replay_recovery": "false", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_sync_write": "false", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_allocator": "hybrid", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_async_db_compaction": "true", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T12:22:11.347 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_create": "true", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_db_create": "false", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_db_path": "", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_db_size": "0", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_path": "", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_size": "107374182400", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_wal_create": "false", 2024-09-15T12:22:11.348 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_wal_path": "", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bluefs": "true", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_autotune": "true", 2024-09-15T12:22:11.349 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_size": "0", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_type": "2q", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_clone_cow": "true", 2024-09-15T12:22:11.350 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_mode": "none", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-15T12:22:11.351 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_freelist": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T12:22:11.352 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T12:22:11.353 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fail_eio": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T12:22:11.354 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T12:22:11.355 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_blob_size": "0", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T12:22:11.356 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T12:22:11.357 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_spdk_mem": "512", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_tracing": "false", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T12:22:11.358 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_write_v2": "false", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_write_v2_random": "false", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T12:22:11.359 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_cluster_require_version": "2", 2024-09-15T12:22:11.360 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_require_signatures": "false", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_require_version": "2", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_service_require_signatures": "false", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_service_require_version": "2", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "cephx_sign_messages": "true", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "chdir": "", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_acl_type": "", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_asio_thread_count": "2", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_cache_mid": "0.750000", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_cache_size": "16384", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_caps_release_delay": "5", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_check_pool_perm": "true", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_debug_force_sync_read": "false", 2024-09-15T12:22:11.361 INFO:teuthology.orchestra.run.smithi103.stdout: "client_debug_getattr_caps": "false", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_debug_inject_features": "", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_die_on_failed_remount": "false", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_dirsize_rbytes": "true", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_force_lazyio": "false", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_fs": "", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_inject_release_failure": "false", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_max_inline_size": "4096", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T12:22:11.362 INFO:teuthology.orchestra.run.smithi103.stdout: "client_mds_namespace": "", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_metadata": "", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_mount_gid": "-1", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_mount_timeout": "300", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_mount_uid": "-1", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_mountpoint": "/", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_notify_timeout": "10", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc": "true", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc_max_objects": "1000", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc_size": "209715200", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T12:22:11.363 INFO:teuthology.orchestra.run.smithi103.stdout: "client_permissions": "true", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_quota": "true", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_quota_df": "true", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_readahead_max_bytes": "0", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_readahead_max_periods": "4", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_readahead_min": "131072", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_reconnect_stale": "false", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_shutdown_timeout": "30", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_snapdir": ".snap", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_tick_interval": "1", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_trace": "", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T12:22:11.364 INFO:teuthology.orchestra.run.smithi103.stdout: "client_use_faked_inos": "false", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "client_use_random_mds": "false", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_graylog": "false", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_graylog_port": "12201", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_monitors": "default=true", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_syslog": "false", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "clog_to_syslog_level": "info", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_addr": "-", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_network": "", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_network_interface": "", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "compressor_zlib_isal": "false", 2024-09-15T12:22:11.365 INFO:teuthology.orchestra.run.smithi103.stdout: "compressor_zlib_level": "5", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "compressor_zstd_level": "1", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_location": "", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_location_hook": "", 2024-09-15T12:22:11.366 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_location_hook_timeout": "10", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "daemonize": "false", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "daos_pool": "tank", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_alienstore": "0/5", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_asok": "1/5", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_asok_assert_abort": "false", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_auth": "20/20", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_bdev": "1/3", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_bluefs": "1/5", 2024-09-15T12:22:11.367 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_bluestore": "1/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_buffer": "0/1", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_ceph_dedup": "0/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_cephsqlite": "0/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_client": "0/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_compressor": "1/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_context": "0/1", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_crush": "1/1", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_crypto": "1/5", 2024-09-15T12:22:11.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_cyanstore": "0/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_dpdk": "1/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_eventtrace": "1/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_filer": "0/1", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_filestore": "1/3", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_finisher": "1/1", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_fuse": "1/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_javaclient": "1/5", 2024-09-15T12:22:11.369 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_journal": "1/3", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_journaler": "0/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_kstore": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_lockdep": "0/1", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mclock": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_balancer": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_locker": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_log": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_migrator": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_memstore": "1/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mgr": "2/5", 2024-09-15T12:22:11.370 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mgrc": "1/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_mon": "20/20", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_monc": "0/10", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_ms": "1/1", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_none": "0/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_objclass": "0/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_objectcacher": "0/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_objecter": "0/1", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_optracker": "0/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_osd": "1/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_paxos": "20/20", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_perfcounter": "1/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_prioritycache": "1/5", 2024-09-15T12:22:11.371 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rados": "0/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rbd": "0/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rbd_replay": "0/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_refs": "0/0", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_reserver": "1/1", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_access": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_flight": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_notification": "1/5", 2024-09-15T12:22:11.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rgw_sync": "1/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_rocksdb": "4/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_backref": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_cache": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_device": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_epm": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_journal": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_lba": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_odata": "0/5", 2024-09-15T12:22:11.373 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_omap": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_onode": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_t": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_seastore_tm": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_striper": "0/1", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_test": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_throttle": "1/1", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_timer": "0/1", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_tp": "0/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "debug_trace": "1/5", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "device_failure_prediction_mode": "none", 2024-09-15T12:22:11.374 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "err_to_graylog": "false", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "err_to_journald": "false", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "err_to_stderr": "true", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "err_to_syslog": "false", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "event_tracing": "false", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_cert_file": "", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_http_port": "9926", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_key_file": "", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_prio_limit": "5", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_sort_metrics": "true", 2024-09-15T12:22:11.375 INFO:teuthology.orchestra.run.smithi103.stdout: "exporter_stats_period": "5", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "fake_statfs_for_testing": "0", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "fatal_signal_handlers": "true", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filer_max_purge_ops": "10", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filer_max_truncate_ops": "128", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_blackhole": "false", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_btrfs_snap": "true", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_caller_concurrency": "10", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T12:22:11.376 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_debug_omap_check": "false", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_debug_verify_split": "false", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_dump_file": "", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fadvise": "true", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fail_eio": "true", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fd_cache_size": "128", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fiemap": "false", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T12:22:11.377 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_inject_stall": "0", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_journal_parallel": "false", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_journal_trailing": "false", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_journal_writeahead": "false", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_kill_at": "0", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T12:22:11.378 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_merge_threshold": "-10", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_odsync_write": "false", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_omap_backend_path": "", 2024-09-15T12:22:11.379 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_op_threads": "2", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_punch_hole": "false", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T12:22:11.380 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_queue_max_ops": "50", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_seek_data_hole": "false", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_sloppy_crc": "false", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_splice": "false", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_split_multiple": "2", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_split_rand_factor": "20", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_update_to": "1000", 2024-09-15T12:22:11.381 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_xfs_extsize": "false", 2024-09-15T12:22:11.382 INFO:teuthology.orchestra.run.smithi103.stdout: "filestore_zfs_snap": "false", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fio_dir": "/tmp/fio", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fsid": "0089f1fe-9ae6-4cff-ad25-a1642089e108", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_allow_other": "true", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_big_writes": "true", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_debug": "false", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_default_permissions": "false", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_disable_pagecache": "false", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_max_write": "0", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_multithreaded": "true", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_require_active_mds": "true", 2024-09-15T12:22:11.383 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_set_user_groups": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_splice_move": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_splice_read": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_splice_write": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "gss_target_name": "ceph", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "heartbeat_file": "", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "heartbeat_inject_failure": "0", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "heartbeat_interval": "5", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "host": "smithi103", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T12:22:11.384 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "inject_early_sigterm": "false", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "jaeger_agent_port": "6799", 2024-09-15T12:22:11.385 INFO:teuthology.orchestra.run.smithi103.stdout: "jaeger_tracing_enable": "false", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_aio": "true", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_align_min_size": "65536", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_block_align": "true", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_block_size": "4096", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_dio": "true", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_discard": "false", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_force_aio": "false", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_ignore_corruption": "false", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_max_write_entries": "100", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_replay_from": "0", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T12:22:11.386 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_write_header_frequency": "0", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journal_zero_on_create": "false", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journaler_prefetch_periods": "10", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journaler_prezero_periods": "5", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "journaler_write_head_interval": "15", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "key": "", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "keyfile": "", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_backend": "rocksdb", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_max_bytes": "67108864", 2024-09-15T12:22:11.387 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_max_ops": "512", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_onode_map_size": "1024", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "kstore_sync_transaction": "false", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "librados_thread_count": "2", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "lockdep": "false", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "lockdep_force_backtrace": "false", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_coarse_timestamps": "true", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_flush_on_exit": "false", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_graylog_port": "12201", 2024-09-15T12:22:11.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_max_new": "1000", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_max_recent": "10000", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_stderr_prefix": "", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": "true", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_graylog": "false", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_journald": "false", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_stderr": "false", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_syslog": "false", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_action_on_write_error": "1", 2024-09-15T12:22:11.389 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_alternate_name_max": "8192", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_asio_thread_count": "2", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_export_pin": "true", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_interval": "10", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_max": "-1", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_max_until": "-1", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_merge_size": "50", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T12:22:11.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_mode": "0", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_split_bits": "3", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_split_size": "10000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T12:22:11.391 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_mid": "0.700000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T12:22:11.392 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_trim_interval": "1", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_debug_auth_pins": "false", 2024-09-15T12:22:11.393 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_debug_frag": "false", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_debug_scatterstat": "false", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_debug_subtrees": "false", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_default_dir_hash": "2", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_defer_session_stale": "true", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dir_max_entries": "0", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dir_prefetch": "true", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T12:22:11.394 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_early_reply": "true", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_enable_op_tracker": "true", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_enforce_unique_name": "true", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T12:22:11.395 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_health_dummy": "false", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T12:22:11.396 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_join_fs": "", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_journal_format": "1", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_create_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_dirfrag_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_export_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_import_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_journal_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_link_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T12:22:11.397 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_openc_at": "0", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_rename_at": "0", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_max_events": "-1", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_max_segments": "128", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_pause": "false", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_segment_size": "0", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T12:22:11.398 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_trim_threshold": "128", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_completed_requests": "100000", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_export_size": "20971520", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_file_recover": "32", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_purge_files": "64", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_purge_ops": "8192", 2024-09-15T12:22:11.399 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_metrics_update_interval": "2", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_min_caps_per_client": "100", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_numa_node": "-1", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_history_duration": "600", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_history_size": "20", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T12:22:11.400 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_op_log_threshold": "5", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_ping_grace": "15", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_ping_interval": "5", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_max_caps": "30000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_replay_interval": "1.000000", 2024-09-15T12:22:11.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_root_ino_gid": "0", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_root_ino_uid": "0", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_scrub_stats_review_period": "1", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T12:22:11.402 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_shutdown_check": "0", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_skip_ino": "0", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_snap_min_uid": "0", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_snap_rstat": "false", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_symlink_recovery": "true", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_thrash_exports": "0", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_thrash_fragments": "0", 2024-09-15T12:22:11.403 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_tick_interval": "5.000000", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_valgrind_exit": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_verify_backtrace": "1", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_verify_scatter": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_wipe_sessions": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool_debug": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "memstore_page_set": "false", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "memstore_page_size": "65536", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_client_bytes": "134217728", 2024-09-15T12:22:11.404 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_client_messages": "512", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_disabled_modules": "", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_max_pg_creating": "1024", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_mds_messages": "128", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T12:22:11.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_mon_messages": "128", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_history_duration": "600", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_history_size": "20", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_op_log_threshold": "5", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_osd_messages": "8192", 2024-09-15T12:22:11.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_pool": "true", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_standby_modules": "true", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_stats_period": "5", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_stats_threshold": "5", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_tick_period": "2", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_allow_pool_delete": "true", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T12:22:11.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_bytes": "104857600", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_directed_command_retry": "2", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_client_target_rank": "-1", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T12:22:11.408 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_level": "debug", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_compact_on_start": "false", 2024-09-15T12:22:11.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_compact_on_trim": "true", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_cpu_threads": "4", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_data_avail_crit": "5", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_data_avail_warn": "5", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_dump_json": "false", 2024-09-15T12:22:11.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_extra_checks": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_down_uptime_grace": "60", 2024-09-15T12:22:11.411 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_election_default_strategy": "1", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_election_timeout": "5.000000", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_enable_op_tracker": "true", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_fake_pool_delete": "false", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_force_quorum_join": "false", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_log_update_period": "5", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_max_detail": "50", 2024-09-15T12:22:11.412 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_to_clog": "true", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_host": "172.21.15.103", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_host_override": "", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_initial_members": "", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_lease": "5.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T12:22:11.413 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_log_full_interval": "50", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_log_max": "10000", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_log_max_summary": "50", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_log_epochs": "500", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_osd": "10000", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T12:22:11.414 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_memory_autotune": "true", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_memory_target": "2147483648", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_digest_period": "5", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_min_osdmap_epochs": "50", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_complaint_time": "30", 2024-09-15T12:22:11.415 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_history_duration": "600", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_history_size": "20", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_op_log_threshold": "5", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T12:22:11.416 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_cache_size": "500", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T12:22:11.417 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_report_timeout": "900", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T12:22:11.418 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T12:22:11.419 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_reweight_max_osds": "4", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_scrub_interval": "86400", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_scrub_max_keys": "100", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_scrub_timeout": "300", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_session_timeout": "300", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_smart_report_timeout": "5", 2024-09-15T12:22:11.420 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_stretch_pool_size": "4", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_debug": "false", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_tick_interval": "5", 2024-09-15T12:22:11.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_older_version": "true", 2024-09-15T12:22:11.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "monmap": "", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T12:22:11.423 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "motr_tracing_enabled": "false", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_op_threads": "3", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_cm": "false", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_device_name": "", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T12:22:11.424 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_sl": "3", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_rdma_type": "ib", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_async_reap_threshold": "5", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_before_connect": "false", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_ipv4": "true", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_ipv6": "false", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_msgr1": "true", 2024-09-15T12:22:11.425 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_msgr2": "true", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_port_max": "7568", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_port_min": "6800", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_retry_count": "3", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_bind_retry_delay": "5", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_blackhole_client": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_blackhole_mds": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_blackhole_mgr": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_blackhole_mon": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_blackhole_osd": "false", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_client_mode": "crc", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_cluster_mode": "crc", 2024-09-15T12:22:11.426 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_cluster_type": "", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_compress_secure": "false", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_crc_data": "true", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_crc_header": "true", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_die_on_bug": "true", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_die_on_old_message": "true", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T12:22:11.427 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_hugepages": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_lro": "true", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_pmd": "", 2024-09-15T12:22:11.428 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_port_id": "0", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_dump_on_send": "false", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_delay_type": "", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_network_congestion": "0", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_inject_socket_failures": "0", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_max_accept_failures": "4", 2024-09-15T12:22:11.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_max_backoff": "15.000000", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_mon_client_mode": "crc", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_mon_cluster_mode": "crc", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_mon_service_mode": "crc", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_osd_compress_mode": "none", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_pq_min_cost": "65536", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_public_type": "", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_service_mode": "crc", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_tcp_nodelay": "true", 2024-09-15T12:22:11.430 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "ms_type": "async+posix", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "no_config_file": "false", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_inflight_ops": "1024", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T12:22:11.431 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter_timeout": "10.000000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "objectstore_blackhole": "false", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "openssl_engine_opts": "", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_max_ops": "4", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_agent_slop": "0.020000", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T12:22:11.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_asio_thread_count": "2", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_async_recovery_min_cost": "100", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backfill_scan_max": "512", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backfill_scan_min": "64", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backoff_on_peering": "false", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_beacon_report_interval": "300", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bench_duration": "30", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T12:22:11.433 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_blkin_trace_all": "false", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_class_update_on_start": "true", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_client_message_cap": "256", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_client_op_priority": "63", 2024-09-15T12:22:11.434 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_client_watch_timeout": "30", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_command_max_records": "256", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_compact_on_start": "false", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_crush_update_on_start": "true", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T12:22:11.435 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_op_order": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T12:22:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_shutdown": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_trim_objects": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T12:22:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_default_notify_timeout": "30", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_ec_partial_reads": "true", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_enable_op_tracker": "true", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T12:22:11.438 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_fast_info": "true", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_fast_shutdown": "true", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_function_tracing": "false", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_grace": "20", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_interval": "6", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T12:22:11.439 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_stale": "600", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_journal_size": "5120", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_kill_backfill_at": "0", 2024-09-15T12:22:11.440 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_map_cache_size": "50", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_map_dedup": "true", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_map_message_max": "40", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_attr_name_len": "100", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_attr_size": "0", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_backfills": "1", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_markdown_count": "5", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_markdown_period": "600", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_object_name_len": "2048", 2024-09-15T12:22:11.441 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_object_size": "134217728", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_pgls": "1024", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_push_cost": "8388608", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_push_objects": "10", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_scrubs": "3", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_max_write_size": "90", 2024-09-15T12:22:11.442 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_profile": "balanced", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T12:22:11.443 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_base": "805306368", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_target": "4294967296", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_target_autotune": "false", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T12:22:11.444 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_min_recovery_priority": "0", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mon_report_interval": "5", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_num_cache_shards": "32", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_numa_node": "-1", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T12:22:11.445 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_objecter_finishers": "1", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_objectstore": "bluestore", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_objectstore_fuse": "false", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_objectstore_tracing": "false", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_history_duration": "600", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_history_size": "20", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_log_threshold": "5", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_shards": "0", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T12:22:11.446 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_op_thread_timeout": "15", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_open_classes_on_start": "true", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_os_flags": "0", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_peering_op_priority": "255", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_delete_priority": "5", 2024-09-15T12:22:11.447 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T12:22:11.448 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_crimson": "true", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_flags": "0", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_min_size": "2", 2024-09-15T12:22:11.449 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_size": "3", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_default_type": "replicated", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T12:22:11.450 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_cost": "20971520", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_active": "0", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_op_priority": "3", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_priority": "5", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T12:22:11.451 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_repair_during_recovery": "false", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T12:22:11.452 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_cost": "52428800", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_end_hour": "0", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T12:22:11.453 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_priority": "5", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_retry_delay": "30", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_retry_trimming": "10", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T12:22:11.454 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_skip_data_digest": "false", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_smart_report_timeout": "5", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_priority": "5", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T12:22:11.455 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_target_transaction_size": "30", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_tracing": "false", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_use_stale_snap": "false", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T12:22:11.456 INFO:teuthology.orchestra.run.smithi103.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_kill_at": "0", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_max_join_drift": "10", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_min": "500", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_min_wait": "0.050000", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_service_trim_max": "500", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_service_trim_min": "10", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_stash_full_interval": "25", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_trim_max": "500", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "paxos_trim_min": "250", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "perf": "true", 2024-09-15T12:22:11.457 INFO:teuthology.orchestra.run.smithi103.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "public_addr": "-", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "public_addrv": "", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "public_bind_addr": "-", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "public_network": "", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "public_network_interface": "", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "qat_compressor_enabled": "false", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "rados_mon_op_timeout": "0", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "rados_osd_op_timeout": "0", 2024-09-15T12:22:11.458 INFO:teuthology.orchestra.run.smithi103.stdout: "rados_tracing": "false", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_atime_update_interval": "60", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache": "true", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_policy": "writearound", 2024-09-15T12:22:11.459 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_size": "33554432", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_compression_hint": "none", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_clone_format": "auto", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_data_pool": "", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_features": "61", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_format": "2", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_map_options": "", 2024-09-15T12:22:11.460 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_order": "22", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_pool": "rbd", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_stripe_count": "0", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_io_scheduler": "simple", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T12:22:11.461 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_order": "24", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_pool": "", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_journal_splay_width": "4", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T12:22:11.462 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T12:22:11.463 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T12:22:11.464 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_op_threads": "1", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T12:22:11.465 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_plugins": "", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T12:22:11.466 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T12:22:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_tracing": "false", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_validate_names": "true", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_validate_pool": "true", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "restapi_base_url": "", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "restapi_log_level": "", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_admin_entry": "admin", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T12:22:11.468 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_asio_assert_yielding": "false", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_backend_store": "rados", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_barbican_url": "", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_beast_enable_async": "true", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T12:22:11.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_cache_enabled": "true", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_config_store": "rados", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_content_length_compat": "false", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_cross_domain_policy": "", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T12:22:11.470 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T12:22:11.471 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T12:22:11.472 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T12:22:11.473 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-15T12:22:11.474 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_log_window": "30", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T12:22:11.475 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_disable_s3select": "false", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T12:22:11.476 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dns_name": "", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dns_s3website_name": "", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T12:22:11.477 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_mdsearch": "true", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_ops_log": "false", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_static_website": "false", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enable_usage_log": "false", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_expose_bucket": "false", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_extended_http_attrs": "", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_filter": "none", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T12:22:11.478 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_objs": "32", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T12:22:11.479 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_init_timeout": "300", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_password": "", 2024-09-15T12:22:11.480 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_project": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_token": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_admin_user": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T12:22:11.481 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_url": "", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_max_objs": "32", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_max_worker": "3", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T12:22:11.482 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ldap_uri": "", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_log_http_headers": "", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T12:22:11.483 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_attr_size": "0", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_listing_results": "5000", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_notify_retries": "10", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T12:22:11.484 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T12:22:11.485 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T12:22:11.486 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_num_control_oids": "8", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_numa_node": "-1", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_op_tracing": "false", 2024-09-15T12:22:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_opa_token": "", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_opa_url": "", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ops_log_rados": "false", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T12:22:11.488 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_print_continue": "true", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T12:22:11.489 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_rados_tracing": "false", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_realm": "", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_realm_id": "", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_region": "", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_request_uri": "", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T12:22:11.490 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_resolve_cname": "false", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_run_sync_thread": "true", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T12:22:11.491 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_script_uri": "", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_service_provider_name": "", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_client_id": "", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_client_secret": "", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_entry": "sts", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_key": "sts", 2024-09-15T12:22:11.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_auth_url": "", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_custom_header": "", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_need_stats": "true", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_tenant_name": "", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_url": "", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T12:22:11.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_lease_period": "120", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_thread_pool_size": "512", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T12:22:11.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_comment": "", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_createby": "", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_encoding": "", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_flag": "false", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_origin": "", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_torrent_tracker": "", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T12:22:11.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_usage_max_shards": "32", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_use_opa_authz": "false", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_counters_cache": "false", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T12:22:11.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_user_unique_email": "true", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_verify_ssl": "true", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zone": "", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zone_id": "", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zonegroup": "", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zonegroup_id": "", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_block_size": "4096", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T12:22:11.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T12:22:11.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_partition_filters": "false", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_perf": "false", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "run_dir": "/var/run/ceph", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_block_create": "true", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T12:22:11.499 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_device_size": "53687091200", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_full_integrity_check": "false", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_main_device_type": "SSD", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T12:22:11.500 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "seastore_segment_size": "67108864", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "set_keepcaps": "false", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "setgroup": "", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "setuser": "", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "setuser_match_path": "", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T12:22:11.501 INFO:teuthology.orchestra.run.smithi103.stdout: "thp": "false", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "threadpool_default_timeout": "60", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "throttler_perf_counter": "true", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tmp_dir": "/tmp", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "uadk_compressor_enabled": "false", 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-15T12:22:11.502 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:22:11.504 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T12:22:11.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.607+0000 7ff9ae791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.613+0000 7ff9ae791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.629 INFO:teuthology.orchestra.run.smithi103.stdout:{} 2024-09-15T12:22:11.629 INFO:teuthology.orchestra.run.smithi103.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T12:22:11.640 DEBUG:teuthology.orchestra.run.smithi103:> type ceph-objectstore-tool 2024-09-15T12:22:11.654 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T12:22:11.654 DEBUG:teuthology.orchestra.run.smithi117:> type ceph-objectstore-tool 2024-09-15T12:22:11.673 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T12:22:11.673 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-15T12:22:11.685 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-15T12:22:11.687 INFO:tasks.radosbench:Beginning radosbench... 2024-09-15T12:22:11.687 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-15T12:22:11.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-15T12:22:11.687 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T12:22:11.687 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-15T12:22:11.688 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:11.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:11.688 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T12:22:11.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:22:11.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.802+0000 7fe160d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.802+0000 7f7644391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.808+0000 7fe160d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.802+0000 7f477e351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.810+0000 7f7644391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:11.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:11.816+0000 7f477e351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:12.015 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:12.016 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:12.029 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-15T12:22:12.001042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":108064,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377379296,"statfs":{"total":386547056640,"available":386436399104,"internally_reserved":0,"allocated":704512,"data_stored":1907549,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.461416"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:07.061503+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062312+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:07.062312+0000","last_peered":"2024-09-15T12:22:07.062312+0000","last_clean":"2024-09-15T12:22:07.062312+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:07.062312+0000","last_undegraded":"2024-09-15T12:22:07.062312+0000","last_fullsized":"2024-09-15T12:22:07.062312+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:07.062090+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.062168+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:07.062168+0000","last_peered":"2024-09-15T12:22:07.062168+0000","last_clean":"2024-09-15T12:22:07.062168+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:07.062168+0000","last_undegraded":"2024-09-15T12:22:07.062168+0000","last_fullsized":"2024-09-15T12:22:07.062168+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063698+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:07.063698+0000","last_peered":"2024-09-15T12:22:07.063698+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:07.063698+0000","last_undegraded":"2024-09-15T12:22:07.063698+0000","last_fullsized":"2024-09-15T12:22:07.063698+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:07.058522+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.065018+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:07.065018+0000","last_peered":"2024-09-15T12:22:07.065018+0000","last_clean":"2024-09-15T12:22:07.065018+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:07.065018+0000","last_undegraded":"2024-09-15T12:22:07.065018+0000","last_fullsized":"2024-09-15T12:22:07.065018+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:07.063684+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:07.058645+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27036,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344804,"statfs":{"total":96636764160,"available":96609079296,"internally_reserved":0,"allocated":192512,"data_stored":624484,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:12.289 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_0' created 2024-09-15T12:22:12.311 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-15T12:22:12.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:12.436+0000 7fc63b791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:12.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:12.449+0000 7fc63b791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:13.294 INFO:teuthology.orchestra.run.smithi103.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-15T12:22:13.940 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:22:14.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:14.046+0000 7facb4391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:14.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:14.051+0000 7facb4391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:14.316 DEBUG:teuthology.orchestra.run.smithi103:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-15T12:22:14.318 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-15T12:22:14.319 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-15T12:22:14.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:hints = 1 2024-09-15T12:22:14.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-15T12:22:14.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object prefix: benchmark_data_smithi103_30172 2024-09-15T12:22:14.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:22:14.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 0 0 0 0 0 0 - 0 2024-09-15T12:22:15.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1 16 208 192 11.9993 12 0.112463 0.0795213 2024-09-15T12:22:16.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:22:16.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:16.297+0000 7f70a5391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:16.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:16.306+0000 7f70a5391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:16.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2 16 405 389 12.1554 12.3125 0.0453141 0.0803777 2024-09-15T12:22:17.030 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-15T12:22:17.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:17.031 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3] 2024-09-15T12:22:17.031 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:22:17.031 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:22:17.099 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:22:17.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3 16 544 528 10.999 8.6875 0.110925 0.079657 2024-09-15T12:22:18.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 4 16 544 528 8.24914 0 - 0.079657 2024-09-15T12:22:18.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:22:18.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:18.800+0000 7fcd59ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:18.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:18.805+0000 7fcd59ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:19.382 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 5 16 544 528 6.59925 0 - 0.079657 2024-09-15T12:22:20.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 6 16 544 528 5.49937 0 - 0.079657 2024-09-15T12:22:21.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 7 16 544 528 4.71372 0 - 0.079657 2024-09-15T12:22:22.100 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:22:22.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:22.101 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 2, 3] 2024-09-15T12:22:22.101 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:22:22.101 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:22:22.169 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:22:22.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 8 16 544 528 4.1245 0 - 0.079657 2024-09-15T12:22:23.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 9 16 544 528 3.66621 0 - 0.079657 2024-09-15T12:22:24.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 10 16 544 528 3.29958 0 - 0.079657 2024-09-15T12:22:25.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 11 16 544 528 2.99961 0 - 0.079657 2024-09-15T12:22:26.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 12 16 544 528 2.74964 0 - 0.079657 2024-09-15T12:22:27.169 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1, 3] live_osds: [0, 2] 2024-09-15T12:22:27.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:27.170 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:22:27.170 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:22:27.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:22:27.173 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:22:27.173 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:22:27.285 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:22:27.288 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:27.288 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:22:27.383 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 13 16 544 528 2.53813 0 - 0.079657 2024-09-15T12:22:28.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 14 16 544 528 2.35684 0 - 0.079657 2024-09-15T12:22:29.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 15 16 544 528 2.19972 0 - 0.079657 2024-09-15T12:22:30.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 16 16 544 528 2.06225 0 - 0.079657 2024-09-15T12:22:31.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 17 16 544 528 1.94094 0 - 0.079657 2024-09-15T12:22:32.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:22:32.318 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:22:32.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 18 16 544 528 1.83311 0 - 0.079657 2024-09-15T12:22:32.429 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:22:32.429 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:22:32.429 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:22:33.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 19 16 544 528 1.73663 0 - 0.079657 2024-09-15T12:22:34.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:22:34.384095+0000 min lat: 0.0205042 max lat: 0.171729 avg lat: 0.079657 2024-09-15T12:22:34.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:22:34.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 20 16 544 528 1.64981 0 - 0.079657 2024-09-15T12:22:35.384 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 21 16 547 531 1.58017 0.0104167 17.4126 0.177732 2024-09-15T12:22:36.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 22 16 547 531 1.50834 0 - 0.177732 2024-09-15T12:22:37.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 23 16 547 531 1.44276 0 - 0.177732 2024-09-15T12:22:37.438 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:22:37.438 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:37.438 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-15T12:22:37.438 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:22:37.439 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:22:37.441 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:22:37.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:22:37.550 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:22:37.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:37.554 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:22:38.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 24 16 547 531 1.38265 0 - 0.177732 2024-09-15T12:22:39.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 25 16 547 531 1.32734 0 - 0.177732 2024-09-15T12:22:40.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 26 16 547 531 1.27628 0 - 0.177732 2024-09-15T12:22:41.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 27 16 547 531 1.22901 0 - 0.177732 2024-09-15T12:22:42.385 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 28 16 547 531 1.18512 0 - 0.177732 2024-09-15T12:22:42.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:22:42.664 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:22:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [] 2024-09-15T12:22:42.665 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:22:43.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 29 16 547 531 1.14425 0 - 0.177732 2024-09-15T12:22:43.866 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:22:43.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:22:44.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 30 16 547 531 1.10611 0 - 0.177732 2024-09-15T12:22:45.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 31 16 626 610 1.22968 0.49375 0.046444 0.811989 2024-09-15T12:22:45.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:22:46.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:46.010+0000 7f84f9b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:46.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:46.021+0000 7f84f9b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:46.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 32 16 845 829 1.61894 13.6875 0.0583805 0.616958 2024-09-15T12:22:47.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 33 16 1050 1034 1.95809 12.8125 0.0410927 0.509912 2024-09-15T12:22:47.677 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:22:47.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:47.677 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:47.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:47.831+0000 7f96fa591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:47.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:47.842+0000 7f96fa591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:48.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:48.101 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:48.123 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-15T12:22:48.008403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19202611,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":365,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":111020,"kb_used_data":3676,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377376340,"statfs":{"total":386547056640,"available":386433372160,"internally_reserved":0,"allocated":3764224,"data_stored":50098201,"data_compressed":2376607,"data_compressed_allocated":3133440,"data_compressed_original":49889280,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001027"},"pg_stats":[{"pgid":"3.f","version":"18'31","reported_seq":43,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.952957+0000","last_change":"2024-09-15T12:22:13.426094+0000","last_active":"2024-09-15T12:22:16.952957+0000","last_peered":"2024-09-15T12:22:16.952957+0000","last_clean":"2024-09-15T12:22:16.952957+0000","last_became_active":"2024-09-15T12:22:13.419469+0000","last_became_peered":"2024-09-15T12:22:13.419469+0000","last_unstale":"2024-09-15T12:22:16.952957+0000","last_undegraded":"2024-09-15T12:22:16.952957+0000","last_fullsized":"2024-09-15T12:22:16.952957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"18'43","reported_seq":55,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.832747+0000","last_change":"2024-09-15T12:22:13.424728+0000","last_active":"2024-09-15T12:22:16.832747+0000","last_peered":"2024-09-15T12:22:16.832747+0000","last_clean":"2024-09-15T12:22:16.832747+0000","last_became_active":"2024-09-15T12:22:13.414053+0000","last_became_peered":"2024-09-15T12:22:13.414053+0000","last_unstale":"2024-09-15T12:22:16.832747+0000","last_undegraded":"2024-09-15T12:22:16.832747+0000","last_fullsized":"2024-09-15T12:22:16.832747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"18'31","reported_seq":43,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.706915+0000","last_change":"2024-09-15T12:22:13.441368+0000","last_active":"2024-09-15T12:22:16.706915+0000","last_peered":"2024-09-15T12:22:16.706915+0000","last_clean":"2024-09-15T12:22:16.706915+0000","last_became_active":"2024-09-15T12:22:13.424957+0000","last_became_peered":"2024-09-15T12:22:13.424957+0000","last_unstale":"2024-09-15T12:22:16.706915+0000","last_undegraded":"2024-09-15T12:22:16.706915+0000","last_fullsized":"2024-09-15T12:22:16.706915+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"18'41","reported_seq":53,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:17.003105+0000","last_change":"2024-09-15T12:22:13.449256+0000","last_active":"2024-09-15T12:22:17.003105+0000","last_peered":"2024-09-15T12:22:17.003105+0000","last_clean":"2024-09-15T12:22:17.003105+0000","last_became_active":"2024-09-15T12:22:13.437101+0000","last_became_peered":"2024-09-15T12:22:13.437101+0000","last_unstale":"2024-09-15T12:22:17.003105+0000","last_undegraded":"2024-09-15T12:22:17.003105+0000","last_fullsized":"2024-09-15T12:22:17.003105+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"18'35","reported_seq":47,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.954748+0000","last_change":"2024-09-15T12:22:13.445248+0000","last_active":"2024-09-15T12:22:16.954748+0000","last_peered":"2024-09-15T12:22:16.954748+0000","last_clean":"2024-09-15T12:22:16.954748+0000","last_became_active":"2024-09-15T12:22:13.430890+0000","last_became_peered":"2024-09-15T12:22:13.430890+0000","last_unstale":"2024-09-15T12:22:16.954748+0000","last_undegraded":"2024-09-15T12:22:16.954748+0000","last_fullsized":"2024-09-15T12:22:16.954748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:42.646735+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:32.314767+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.317617+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:14.317617+0000","last_peered":"2024-09-15T12:22:14.317617+0000","last_clean":"2024-09-15T12:22:14.317617+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:14.317617+0000","last_undegraded":"2024-09-15T12:22:14.317617+0000","last_fullsized":"2024-09-15T12:22:14.317617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:32.314771+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":27,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.312592+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:14.312592+0000","last_peered":"2024-09-15T12:22:14.312592+0000","last_clean":"2024-09-15T12:22:14.312592+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:14.312592+0000","last_undegraded":"2024-09-15T12:22:14.312592+0000","last_fullsized":"2024-09-15T12:22:14.312592+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"18'33","reported_seq":45,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.801226+0000","last_change":"2024-09-15T12:22:13.413767+0000","last_active":"2024-09-15T12:22:16.801226+0000","last_peered":"2024-09-15T12:22:16.801226+0000","last_clean":"2024-09-15T12:22:16.801226+0000","last_became_active":"2024-09-15T12:22:13.403838+0000","last_became_peered":"2024-09-15T12:22:13.403838+0000","last_unstale":"2024-09-15T12:22:16.801226+0000","last_undegraded":"2024-09-15T12:22:16.801226+0000","last_fullsized":"2024-09-15T12:22:16.801226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.319016+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:14.319016+0000","last_peered":"2024-09-15T12:22:14.319016+0000","last_clean":"2024-09-15T12:22:14.319016+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:14.319016+0000","last_undegraded":"2024-09-15T12:22:14.319016+0000","last_fullsized":"2024-09-15T12:22:14.319016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"18'30","reported_seq":42,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:17.007128+0000","last_change":"2024-09-15T12:22:13.460906+0000","last_active":"2024-09-15T12:22:17.007128+0000","last_peered":"2024-09-15T12:22:17.007128+0000","last_clean":"2024-09-15T12:22:17.007128+0000","last_became_active":"2024-09-15T12:22:13.452045+0000","last_became_peered":"2024-09-15T12:22:13.452045+0000","last_unstale":"2024-09-15T12:22:17.007128+0000","last_undegraded":"2024-09-15T12:22:17.007128+0000","last_fullsized":"2024-09-15T12:22:17.007128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:42.646737+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.316783+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:14.316783+0000","last_peered":"2024-09-15T12:22:14.316783+0000","last_clean":"2024-09-15T12:22:14.316783+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:14.316783+0000","last_undegraded":"2024-09-15T12:22:14.316783+0000","last_fullsized":"2024-09-15T12:22:14.316783+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"18'40","reported_seq":52,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.965720+0000","last_change":"2024-09-15T12:22:13.427939+0000","last_active":"2024-09-15T12:22:16.965720+0000","last_peered":"2024-09-15T12:22:16.965720+0000","last_clean":"2024-09-15T12:22:16.965720+0000","last_became_active":"2024-09-15T12:22:13.419108+0000","last_became_peered":"2024-09-15T12:22:13.419108+0000","last_unstale":"2024-09-15T12:22:16.965720+0000","last_undegraded":"2024-09-15T12:22:16.965720+0000","last_fullsized":"2024-09-15T12:22:16.965720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:32.314773+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18612224,"num_objects":284,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":18176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3010560,"data_stored":48168960,"data_compressed":2286439,"data_compressed_allocated":3010560,"data_compressed_original":48168960,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":24,"acting":24,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28572,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1781760,"data_stored":25670082,"data_compressed":1218384,"data_compressed_allocated":1605632,"data_compressed_original":25608192,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28456,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343384,"statfs":{"total":96636764160,"available":96607625216,"internally_reserved":0,"allocated":1662976,"data_stored":23769538,"data_compressed":1128167,"data_compressed_allocated":1486848,"data_compressed_original":23707648,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":23134208,"data_compressed":1098111,"data_compressed_allocated":1445888,"data_compressed_original":23134208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1564672,"data_stored":25034752,"data_compressed":1188328,"data_compressed_allocated":1564672,"data_compressed_original":25034752,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:48.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 34 16 1244 1228 2.25707 12.125 0.0902865 0.442536 2024-09-15T12:22:48.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:22:48.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:48.587+0000 7f31c8351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:48.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:48.595+0000 7f31c8351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:49.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 35 16 1445 1429 2.55147 12.5625 0.0568531 0.391218 2024-09-15T12:22:50.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 36 16 1648 1632 2.83299 12.6875 0.087834 0.352346 2024-09-15T12:22:50.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:22:51.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:51.087+0000 7fe920f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:51.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:51.097+0000 7fe920f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:51.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 37 16 1848 1832 3.09422 12.5 0.0896315 0.322687 2024-09-15T12:22:52.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 38 16 2055 2039 3.35322 12.9375 0.0696322 0.297902 2024-09-15T12:22:53.125 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:22:53.125 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:53.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:22:53.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:53.274+0000 7f9171351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:53.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:53.288+0000 7f9171351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:53.386 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 39 16 2258 2242 3.59252 12.6875 0.120788 0.278021 2024-09-15T12:22:53.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:22:53.570 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:22:53.571 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:22:53.591 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-15T12:22:52.009305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19202611,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":365,"num_write_kb":19284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":111020,"kb_used_data":3676,"kb_used_omap":0,"kb_used_meta":107264,"kb_avail":377376340,"statfs":{"total":386547056640,"available":386433372160,"internally_reserved":0,"allocated":3764224,"data_stored":50098201,"data_compressed":2376607,"data_compressed_allocated":3133440,"data_compressed_original":49889280,"omap_allocated":0,"internal_metadata":109838336},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.361607"},"pg_stats":[{"pgid":"3.f","version":"18'31","reported_seq":43,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.952957+0000","last_change":"2024-09-15T12:22:13.426094+0000","last_active":"2024-09-15T12:22:16.952957+0000","last_peered":"2024-09-15T12:22:16.952957+0000","last_clean":"2024-09-15T12:22:16.952957+0000","last_became_active":"2024-09-15T12:22:13.419469+0000","last_became_peered":"2024-09-15T12:22:13.419469+0000","last_unstale":"2024-09-15T12:22:16.952957+0000","last_undegraded":"2024-09-15T12:22:16.952957+0000","last_fullsized":"2024-09-15T12:22:16.952957+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"18'43","reported_seq":55,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.832747+0000","last_change":"2024-09-15T12:22:13.424728+0000","last_active":"2024-09-15T12:22:16.832747+0000","last_peered":"2024-09-15T12:22:16.832747+0000","last_clean":"2024-09-15T12:22:16.832747+0000","last_became_active":"2024-09-15T12:22:13.414053+0000","last_became_peered":"2024-09-15T12:22:13.414053+0000","last_unstale":"2024-09-15T12:22:16.832747+0000","last_undegraded":"2024-09-15T12:22:16.832747+0000","last_fullsized":"2024-09-15T12:22:16.832747+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":2752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"18'31","reported_seq":43,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.706915+0000","last_change":"2024-09-15T12:22:13.441368+0000","last_active":"2024-09-15T12:22:16.706915+0000","last_peered":"2024-09-15T12:22:16.706915+0000","last_clean":"2024-09-15T12:22:16.706915+0000","last_became_active":"2024-09-15T12:22:13.424957+0000","last_became_peered":"2024-09-15T12:22:13.424957+0000","last_unstale":"2024-09-15T12:22:16.706915+0000","last_undegraded":"2024-09-15T12:22:16.706915+0000","last_fullsized":"2024-09-15T12:22:16.706915+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2031616,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"18'41","reported_seq":53,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:17.003105+0000","last_change":"2024-09-15T12:22:13.449256+0000","last_active":"2024-09-15T12:22:17.003105+0000","last_peered":"2024-09-15T12:22:17.003105+0000","last_clean":"2024-09-15T12:22:17.003105+0000","last_became_active":"2024-09-15T12:22:13.437101+0000","last_became_peered":"2024-09-15T12:22:13.437101+0000","last_unstale":"2024-09-15T12:22:17.003105+0000","last_undegraded":"2024-09-15T12:22:17.003105+0000","last_fullsized":"2024-09-15T12:22:17.003105+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":2624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"18'35","reported_seq":47,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.954748+0000","last_change":"2024-09-15T12:22:13.445248+0000","last_active":"2024-09-15T12:22:16.954748+0000","last_peered":"2024-09-15T12:22:16.954748+0000","last_clean":"2024-09-15T12:22:16.954748+0000","last_became_active":"2024-09-15T12:22:13.430890+0000","last_became_peered":"2024-09-15T12:22:13.430890+0000","last_unstale":"2024-09-15T12:22:16.954748+0000","last_undegraded":"2024-09-15T12:22:16.954748+0000","last_fullsized":"2024-09-15T12:22:16.954748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2293760,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":88,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.058645+0000","last_change":"2024-09-15T12:22:03.078386+0000","last_active":"2024-09-15T12:22:07.058645+0000","last_peered":"2024-09-15T12:22:07.058645+0000","last_clean":"2024-09-15T12:22:07.058645+0000","last_became_active":"2024-09-15T12:22:03.070100+0000","last_became_peered":"2024-09-15T12:22:03.070100+0000","last_unstale":"2024-09-15T12:22:42.646735+0000","last_undegraded":"2024-09-15T12:22:07.058645+0000","last_fullsized":"2024-09-15T12:22:07.058645+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":17,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.063684+0000","last_change":"2024-09-15T12:22:07.065887+0000","last_active":"2024-09-15T12:22:07.063684+0000","last_peered":"2024-09-15T12:22:07.063684+0000","last_clean":"2024-09-15T12:22:07.063684+0000","last_became_active":"2024-09-15T12:22:05.127042+0000","last_became_peered":"2024-09-15T12:22:05.127042+0000","last_unstale":"2024-09-15T12:22:32.314767+0000","last_undegraded":"2024-09-15T12:22:07.063684+0000","last_fullsized":"2024-09-15T12:22:07.063684+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.317617+0000","last_change":"2024-09-15T12:22:07.067489+0000","last_active":"2024-09-15T12:22:14.317617+0000","last_peered":"2024-09-15T12:22:14.317617+0000","last_clean":"2024-09-15T12:22:14.317617+0000","last_became_active":"2024-09-15T12:22:05.159794+0000","last_became_peered":"2024-09-15T12:22:05.159794+0000","last_unstale":"2024-09-15T12:22:14.317617+0000","last_undegraded":"2024-09-15T12:22:14.317617+0000","last_fullsized":"2024-09-15T12:22:14.317617+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.058522+0000","last_change":"2024-09-15T12:22:07.060706+0000","last_active":"2024-09-15T12:22:07.058522+0000","last_peered":"2024-09-15T12:22:07.058522+0000","last_clean":"2024-09-15T12:22:07.058522+0000","last_became_active":"2024-09-15T12:22:05.112758+0000","last_became_peered":"2024-09-15T12:22:05.112758+0000","last_unstale":"2024-09-15T12:22:32.314771+0000","last_undegraded":"2024-09-15T12:22:07.058522+0000","last_fullsized":"2024-09-15T12:22:07.058522+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":27,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.312592+0000","last_change":"2024-09-15T12:22:07.063698+0000","last_active":"2024-09-15T12:22:14.312592+0000","last_peered":"2024-09-15T12:22:14.312592+0000","last_clean":"2024-09-15T12:22:14.312592+0000","last_became_active":"2024-09-15T12:22:05.119110+0000","last_became_peered":"2024-09-15T12:22:05.119110+0000","last_unstale":"2024-09-15T12:22:14.312592+0000","last_undegraded":"2024-09-15T12:22:14.312592+0000","last_fullsized":"2024-09-15T12:22:14.312592+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"18'33","reported_seq":45,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.801226+0000","last_change":"2024-09-15T12:22:13.413767+0000","last_active":"2024-09-15T12:22:16.801226+0000","last_peered":"2024-09-15T12:22:16.801226+0000","last_clean":"2024-09-15T12:22:16.801226+0000","last_became_active":"2024-09-15T12:22:13.403838+0000","last_became_peered":"2024-09-15T12:22:13.403838+0000","last_unstale":"2024-09-15T12:22:16.801226+0000","last_undegraded":"2024-09-15T12:22:16.801226+0000","last_fullsized":"2024-09-15T12:22:16.801226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162688,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":2112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.319016+0000","last_change":"2024-09-15T12:22:07.065566+0000","last_active":"2024-09-15T12:22:14.319016+0000","last_peered":"2024-09-15T12:22:14.319016+0000","last_clean":"2024-09-15T12:22:14.319016+0000","last_became_active":"2024-09-15T12:22:05.146276+0000","last_became_peered":"2024-09-15T12:22:05.146276+0000","last_unstale":"2024-09-15T12:22:14.319016+0000","last_undegraded":"2024-09-15T12:22:14.319016+0000","last_fullsized":"2024-09-15T12:22:14.319016+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"18'30","reported_seq":42,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:17.007128+0000","last_change":"2024-09-15T12:22:13.460906+0000","last_active":"2024-09-15T12:22:17.007128+0000","last_peered":"2024-09-15T12:22:17.007128+0000","last_clean":"2024-09-15T12:22:17.007128+0000","last_became_active":"2024-09-15T12:22:13.452045+0000","last_became_peered":"2024-09-15T12:22:13.452045+0000","last_unstale":"2024-09-15T12:22:17.007128+0000","last_undegraded":"2024-09-15T12:22:17.007128+0000","last_fullsized":"2024-09-15T12:22:17.007128+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1966080,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.062090+0000","last_change":"2024-09-15T12:22:07.065598+0000","last_active":"2024-09-15T12:22:07.062090+0000","last_peered":"2024-09-15T12:22:07.062090+0000","last_clean":"2024-09-15T12:22:07.062090+0000","last_became_active":"2024-09-15T12:22:05.113794+0000","last_became_peered":"2024-09-15T12:22:05.113794+0000","last_unstale":"2024-09-15T12:22:42.646737+0000","last_undegraded":"2024-09-15T12:22:07.062090+0000","last_fullsized":"2024-09-15T12:22:07.062090+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:22:12.285969+0000","last_change":"2024-09-15T12:22:12.285969+0000","last_active":"2024-09-15T12:22:12.285969+0000","last_peered":"2024-09-15T12:22:12.285969+0000","last_clean":"2024-09-15T12:22:12.285969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:22:12.285969+0000","last_undegraded":"2024-09-15T12:22:12.285969+0000","last_fullsized":"2024-09-15T12:22:12.285969+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:14.316783+0000","last_change":"2024-09-15T12:22:07.065399+0000","last_active":"2024-09-15T12:22:14.316783+0000","last_peered":"2024-09-15T12:22:14.316783+0000","last_clean":"2024-09-15T12:22:14.316783+0000","last_became_active":"2024-09-15T12:22:05.140400+0000","last_became_peered":"2024-09-15T12:22:05.140400+0000","last_unstale":"2024-09-15T12:22:14.316783+0000","last_undegraded":"2024-09-15T12:22:14.316783+0000","last_fullsized":"2024-09-15T12:22:14.316783+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"18'40","reported_seq":52,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-15T12:22:16.965720+0000","last_change":"2024-09-15T12:22:13.427939+0000","last_active":"2024-09-15T12:22:16.965720+0000","last_peered":"2024-09-15T12:22:16.965720+0000","last_clean":"2024-09-15T12:22:16.965720+0000","last_became_active":"2024-09-15T12:22:13.419108+0000","last_became_peered":"2024-09-15T12:22:13.419108+0000","last_unstale":"2024-09-15T12:22:16.965720+0000","last_undegraded":"2024-09-15T12:22:16.965720+0000","last_fullsized":"2024-09-15T12:22:16.965720+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:12.285969+0000","last_clean_scrub_stamp":"2024-09-15T12:22:12.285969+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":2560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":15,"state":"stale+active+clean","last_fresh":"2024-09-15T12:22:07.061503+0000","last_change":"2024-09-15T12:22:07.063756+0000","last_active":"2024-09-15T12:22:07.061503+0000","last_peered":"2024-09-15T12:22:07.061503+0000","last_clean":"2024-09-15T12:22:07.061503+0000","last_became_active":"2024-09-15T12:22:05.127322+0000","last_became_peered":"2024-09-15T12:22:05.127322+0000","last_unstale":"2024-09-15T12:22:32.314773+0000","last_undegraded":"2024-09-15T12:22:07.061503+0000","last_fullsized":"2024-09-15T12:22:07.061503+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":18612224,"num_objects":284,"num_object_clones":0,"num_object_copies":852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":284,"num_write_kb":18176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3010560,"data_stored":48168960,"data_compressed":2286439,"data_compressed_allocated":3010560,"data_compressed_original":48168960,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":24,"acting":24,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28572,"kb_used_data":1740,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1781760,"data_stored":25670082,"data_compressed":1218384,"data_compressed_allocated":1605632,"data_compressed_original":25608192,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":26960,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344880,"statfs":{"total":96636764160,"available":96609157120,"internally_reserved":0,"allocated":131072,"data_stored":34116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28456,"kb_used_data":1624,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94343384,"statfs":{"total":96636764160,"available":96607625216,"internally_reserved":0,"allocated":1662976,"data_stored":23769538,"data_compressed":1128167,"data_compressed_allocated":1486848,"data_compressed_original":23707648,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1445888,"data_stored":23134208,"data_compressed":1098111,"data_compressed_allocated":1445888,"data_compressed_original":23134208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1564672,"data_stored":25034752,"data_compressed":1188328,"data_compressed_allocated":1564672,"data_compressed_original":25034752,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:22:53.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:53.608+0000 7f6e14791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:53.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:53.615+0000 7f6e14791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:54.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:22:54.386504+0000 min lat: 0.0176454 max lat: 28.1047 avg lat: 0.261731 2024-09-15T12:22:54.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:22:54.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 40 16 2458 2442 3.81517 12.5 0.0848809 0.261731 2024-09-15T12:22:55.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 41 16 2664 2648 4.03611 12.875 0.0573241 0.247537 2024-09-15T12:22:55.984 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:22:56.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:56.135+0000 7f8db0791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:56.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:56.143+0000 7f8db0791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:56.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 42 16 2860 2844 4.23164 12.25 0.0485871 0.235962 2024-09-15T12:22:57.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 43 16 3063 3047 4.42826 12.6875 0.0280008 0.225546 2024-09-15T12:22:58.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 44 16 3264 3248 4.6131 12.5625 0.0259035 0.216567 2024-09-15T12:22:58.532 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:22:58.594 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:22:58.594 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:22:58.594 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:22:58.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:22:58.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.681+0000 7fad72b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:58.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.689+0000 7fad72b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:58.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.759+0000 7f7b04c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.774+0000 7f7b04c6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:22:58.815 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:22:58.816 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:22:58.816 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:22:58.816 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:22:58.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:58.819 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:22:58.820 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:22:58.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.974+0000 7f4e5f0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:58.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:58.990+0000 7f4e5f0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:22:59.071 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:22:59.072 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:22:59.075 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:59.076 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:22:59.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:22:59.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:59.219+0000 7f057cf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:59.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:59.235+0000 7f057cf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:22:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:22:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:22:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:22:59.273 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:22:59.274 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:22:59.278 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:59.278 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:22:59.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:22:59.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 45 16 3456 3440 4.77722 12 0.0610702 0.209002 2024-09-15T12:22:59.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:59.420+0000 7ff5a4111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:59.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:22:59.433+0000 7ff5a4111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:22:59.555 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:22:59.556 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:22:59.558 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:22:59.559 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:23:00.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 46 16 3659 3643 4.94916 12.6875 0.0981261 0.201887 2024-09-15T12:23:01.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:01.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 47 16 3840 3824 5.08452 11.3125 0.0468795 0.196516 2024-09-15T12:23:01.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:01.460+0000 7f0aa5b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:01.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:01.470+0000 7f0aa5b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:02.387 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 48 16 4031 4015 5.22727 11.9375 0.0808129 0.191114 2024-09-15T12:23:03.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 49 16 4224 4208 5.36674 12.0625 0.122614 0.186184 2024-09-15T12:23:03.872 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:04.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:04.028+0000 7f7fa0791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:04.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:04.037+0000 7f7fa0791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:04.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 50 16 4417 4401 5.50063 12.0625 0.0529749 0.181614 2024-09-15T12:23:04.559 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:23:04.560 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:23:04.560 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 2, 1, 3] 2024-09-15T12:23:04.560 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:23:04.560 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:23:04.640 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:23:05.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 51 16 4454 4438 5.43811 2.3125 0.127195 0.180885 2024-09-15T12:23:06.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 52 16 4454 4438 5.33353 0 - 0.180885 2024-09-15T12:23:06.428 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:06.538 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:06.535+0000 7f51e1591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:06.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:06.541+0000 7f51e1591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:07.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 53 16 4454 4438 5.23289 0 - 0.180885 2024-09-15T12:23:08.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 54 16 4454 4438 5.13599 0 - 0.180885 2024-09-15T12:23:09.388 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 55 16 4454 4438 5.0426 0 - 0.180885 2024-09-15T12:23:09.641 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:23:09.641 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:23:09.642 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 1] 2024-09-15T12:23:09.642 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:23:09.642 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:23:09.709 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:23:10.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 56 16 4454 4438 4.95255 0 - 0.180885 2024-09-15T12:23:11.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 57 16 4454 4438 4.86566 0 - 0.180885 2024-09-15T12:23:12.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 58 16 4454 4438 4.78177 0 - 0.180885 2024-09-15T12:23:13.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 59 16 4454 4438 4.70072 0 - 0.180885 2024-09-15T12:23:14.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:23:14.388779+0000 min lat: 0.0168118 max lat: 28.1047 avg lat: 0.180885 2024-09-15T12:23:14.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:23:14.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 60 16 4454 4438 4.62237 0 - 0.180885 2024-09-15T12:23:14.711 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3, 1] live_osds: [0, 2] 2024-09-15T12:23:14.711 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:23:14.711 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:23:14.711 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:23:14.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:14.823+0000 7f09dac6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:14.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:14.832+0000 7f09dac6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:23:14.849 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:23:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:23:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:23:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:23:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:23:14.850 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:23:14.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:23:14.853 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:23:14.853 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:23:14.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:14.956+0000 7fb372191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:14.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:14.965+0000 7fb372191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:15.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 61 16 4454 4438 4.54659 0 - 0.180885 2024-09-15T12:23:16.389 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 62 16 4454 4438 4.47326 0 - 0.180885 2024-09-15T12:23:17.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 63 16 4454 4438 4.40225 0 - 0.180885 2024-09-15T12:23:18.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 64 16 4454 4438 4.33346 0 - 0.180885 2024-09-15T12:23:19.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 65 16 4454 4438 4.26679 0 - 0.180885 2024-09-15T12:23:20.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 66 16 4454 4438 4.20214 0 - 0.180885 2024-09-15T12:23:21.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 67 16 4454 4438 4.13942 0 - 0.180885 2024-09-15T12:23:22.390 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 68 16 4454 4438 4.07855 0 - 0.180885 2024-09-15T12:23:23.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 69 16 4454 4438 4.01943 0 - 0.180885 2024-09-15T12:23:24.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 70 16 4454 4438 3.96201 0 - 0.180885 2024-09-15T12:23:25.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 71 16 4454 4438 3.9062 0 - 0.180885 2024-09-15T12:23:26.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 72 16 4454 4438 3.85195 0 - 0.180885 2024-09-15T12:23:27.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 73 16 4454 4438 3.79918 0 - 0.180885 2024-09-15T12:23:28.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 74 16 4454 4438 3.74784 0 - 0.180885 2024-09-15T12:23:29.391 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 75 16 4454 4438 3.69787 0 - 0.180885 2024-09-15T12:23:30.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 76 16 4454 4438 3.64921 0 - 0.180885 2024-09-15T12:23:30.654 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:30.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:30.659 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:31.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 77 16 4454 4438 3.60182 0 - 0.180885 2024-09-15T12:23:32.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 78 16 4460 4444 3.56045 0.0138889 0.0153759 0.198993 2024-09-15T12:23:32.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:32.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:32.775+0000 7fea61551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:32.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:32.780+0000 7fea61551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:33.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 79 16 4460 4444 3.51538 0 - 0.198993 2024-09-15T12:23:34.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:23:34.391872+0000 min lat: 0.0153759 max lat: 28.1047 avg lat: 0.198993 2024-09-15T12:23:34.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:23:34.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 80 16 4460 4444 3.47144 0 - 0.198993 2024-09-15T12:23:35.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 81 16 4460 4444 3.42858 0 - 0.198993 2024-09-15T12:23:36.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 82 16 4460 4444 3.38676 0 - 0.198993 2024-09-15T12:23:37.392 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 83 16 4465 4449 3.34972 0.0625 0.0228524 0.220763 2024-09-15T12:23:37.788 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:37.809 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:37.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:37.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:38.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 84 16 4465 4449 3.30985 0 - 0.220763 2024-09-15T12:23:39.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 85 16 4465 4449 3.27091 0 - 0.220763 2024-09-15T12:23:39.814 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:39.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:39.929+0000 7fe9ed697640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:39.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:39.935+0000 7fe9ed697640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:39.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:39.979 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:23:39.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:23:39.987 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:39.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:39.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:40.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:40.082+0000 7f508e191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:40.091+0000 7f508e191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:23:40.111 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:23:40.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:23:40.114 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:23:40.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:23:40.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:40.215+0000 7fd858391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:40.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:40.224+0000 7fd858391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:40.230 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:40.234 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:23:40.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 86 16 4465 4449 3.23287 0 - 0.220763 2024-09-15T12:23:41.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 87 16 4465 4449 3.19571 0 - 0.220763 2024-09-15T12:23:41.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:42.113 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:42.109+0000 7f87a7191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:42.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:42.116+0000 7f87a7191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:42.144 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:42.169 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:42.172 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:42.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:42.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 88 16 4465 4449 3.15939 0 - 0.220763 2024-09-15T12:23:43.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 89 16 4465 4449 3.1239 0 - 0.220763 2024-09-15T12:23:44.174 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:44.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:44.289+0000 7f6f0b951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:44.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:44.294+0000 7f6f0b951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:44.323 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:44.347 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:44.351 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:44.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:44.393 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 90 16 4465 4449 3.08918 0 - 0.220763 2024-09-15T12:23:45.234 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3, 1] live_osds: [0, 2] 2024-09-15T12:23:45.235 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:23:45.235 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:23:45.235 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:23:45.235 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:23:45.238 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:23:45.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:23:45.336 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:23:45.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:23:45.340 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:23:45.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 91 16 4465 4449 3.05524 0 - 0.220763 2024-09-15T12:23:46.352 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:46.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 92 16 4465 4449 3.02203 0 - 0.220763 2024-09-15T12:23:46.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:46.458+0000 7f1d57111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:46.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:46.463+0000 7f1d57111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:46.493 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:46.517 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:46.521 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:46.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:47.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 93 16 4465 4449 2.98954 0 - 0.220763 2024-09-15T12:23:48.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 94 16 4465 4449 2.95773 0 - 0.220763 2024-09-15T12:23:48.522 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:48.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:48.628+0000 7fdeb0591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:48.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:48.634+0000 7fdeb0591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:48.663 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:48.688 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:48.691 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:48.692 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:49.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 95 16 4465 4449 2.9266 0 - 0.220763 2024-09-15T12:23:50.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:23:50.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 96 16 4465 4449 2.89612 0 - 0.220763 2024-09-15T12:23:50.440 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:23:50.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:23:50.443 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:23:50.693 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:50.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:50.798+0000 7fb9fc47d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:50.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:50.803+0000 7fb9fc47d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:50.832 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:50.856 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:50.859 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:50.860 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:51.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 97 16 4465 4449 2.86626 0 - 0.220763 2024-09-15T12:23:52.394 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 98 16 4465 4449 2.83701 0 - 0.220763 2024-09-15T12:23:52.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:52.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:52.966+0000 7f933fb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:52.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:52.972+0000 7f933fb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:53.000 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:53.023 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:53.027 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:53.027 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:53.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 99 16 4465 4449 2.80836 0 - 0.220763 2024-09-15T12:23:54.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:23:54.394376+0000 min lat: 0.0153759 max lat: 32.6199 avg lat: 0.220763 2024-09-15T12:23:54.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:23:54.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 100 16 4465 4449 2.78027 0 - 0.220763 2024-09-15T12:23:55.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:55.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:55.137+0000 7f223bcd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:55.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:55.142+0000 7f223bcd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:55.174 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:23:55.201 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:55.206 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:55.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:55.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 101 16 4465 4449 2.75275 0 - 0.220763 2024-09-15T12:23:55.445 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:23:55.545 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:23:55.549 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:23:55.549 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:23:56.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 102 16 4465 4449 2.72576 0 - 0.220763 2024-09-15T12:23:57.207 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:23:57.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:57.312+0000 7f83944bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:57.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:57.318+0000 7f83944bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:57.387 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:23:57.390 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:23:57.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:23:57.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 103 16 4465 4449 2.6993 0 - 0.220763 2024-09-15T12:23:58.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total time run: 103.549 2024-09-15T12:23:58.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total writes made: 4465 2024-09-15T12:23:58.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Write size: 65536 2024-09-15T12:23:58.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object size: 65536 2024-09-15T12:23:58.395 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Bandwidth (MB/sec): 2.69498 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Bandwidth: 5.0616 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max bandwidth (MB/sec): 13.6875 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min bandwidth (MB/sec): 0 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average IOPS: 43 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev IOPS: 80.9914 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max IOPS: 219 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min IOPS: 0 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average Latency(s): 0.370659 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Latency(s): 3.27466 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max latency(s): 53.3055 2024-09-15T12:23:58.396 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min latency(s): 0.0153759 2024-09-15T12:23:58.417 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T12:23:59.391 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:23:59.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:59.561+0000 7fdbfad91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:23:59.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:23:59.570+0000 7fdbfad91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:00.084 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:00.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:00.088 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:00.550 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:24:00.802 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:24:00.802 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777439, 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4229.0:5068", 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "3.a", 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 30, 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 29, 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "3:5a759f15:::benchmark_data_smithi103_30172_object600:head", 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0" 2024-09-15T12:24:00.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777440, 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4229.0:5072", 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "3.9", 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 30, 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 29, 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "3:942b5b5a:::benchmark_data_smithi103_30172_object604:head", 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0" 2024-09-15T12:24:00.804 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777441, 2024-09-15T12:24:00.805 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4229.0:5075", 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "3.9", 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 30, 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 29, 2024-09-15T12:24:00.806 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "3:9e343122:::benchmark_data_smithi103_30172_object607:head", 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0" 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 238, 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.807 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.213379+0000" 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.213397+0000" 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.808 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.214088+0000" 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.216123+0000" 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.216248+0000" 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.811 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.241028+0000" 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.216834+0000" 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.813 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.217622+0000" 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.814 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.218448+0000" 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.818 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.224212+0000" 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.239366+0000" 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.820 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.821 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 257, 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.822 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.344000+0000" 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.344018+0000" 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.823 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.824 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.824 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.344246+0000" 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.344487+0000" 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.826 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.344772+0000" 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.827 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.389131+0000" 2024-09-15T12:24:00.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.345358+0000" 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.829 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.345679+0000" 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.831 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.346427+0000" 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.834 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.354538+0000" 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.382898+0000" 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 260, 2024-09-15T12:24:00.837 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.838 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.838 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.366137+0000" 2024-09-15T12:24:00.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.838 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.366153+0000" 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.370268+0000" 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.370895+0000" 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.841 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.371838+0000" 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.842 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.398651+0000" 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.372759+0000" 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.844 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.374698+0000" 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.845 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.375387+0000" 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.380794+0000" 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.397302+0000" 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 263, 2024-09-15T12:24:00.851 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.394350+0000" 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.394380+0000" 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.397379+0000" 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.398879+0000" 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.399047+0000" 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.450601+0000" 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.399768+0000" 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.400859+0000" 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.401850+0000" 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.861 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.411401+0000" 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.443953+0000" 2024-09-15T12:24:00.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.864 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 267, 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.422221+0000" 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.865 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.422243+0000" 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.422575+0000" 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.422780+0000" 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.423601+0000" 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.494993+0000" 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.869 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.424251+0000" 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.426202+0000" 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.426764+0000" 2024-09-15T12:24:00.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.435746+0000" 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.486689+0000" 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 341, 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.861915+0000" 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.861934+0000" 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.862104+0000" 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.862222+0000" 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.862377+0000" 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.895105+0000" 2024-09-15T12:24:00.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.863847+0000" 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.864568+0000" 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.866092+0000" 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.873406+0000" 2024-09-15T12:24:00.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.892216+0000" 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.893 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 343, 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.896927+0000" 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.896942+0000" 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.897165+0000" 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.895 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.897398+0000" 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.897831+0000" 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.897 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.928519+0000" 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.898584+0000" 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.899309+0000" 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.899 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.901 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.901484+0000" 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.907548+0000" 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.903 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.924968+0000" 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.905 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 352, 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.944867+0000" 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.944885+0000" 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.945039+0000" 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.946214+0000" 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.946440+0000" 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.993761+0000" 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.946942+0000" 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.947530+0000" 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.911 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.948185+0000" 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.915 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.952090+0000" 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.987879+0000" 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.917 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 355, 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.949911+0000" 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.949925+0000" 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.918 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.950151+0000" 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.951134+0000" 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.953015+0000" 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.008242+0000" 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.958357+0000" 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.960791+0000" 2024-09-15T12:24:00.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.966179+0000" 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.926 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.983406+0000" 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.000961+0000" 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.929 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 358, 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.996837+0000" 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.996869+0000" 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.930 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.997130+0000" 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.997285+0000" 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.997695+0000" 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.045742+0000" 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.998178+0000" 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:23:59.998582+0000" 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.000473+0000" 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.937 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.009525+0000" 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.040335+0000" 2024-09-15T12:24:00.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 360, 2024-09-15T12:24:00.941 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.004756+0000" 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.004767+0000" 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.008838+0000" 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.013743+0000" 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.014036+0000" 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.944 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.071870+0000" 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.019272+0000" 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.021331+0000" 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.947 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.024316+0000" 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.949 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.047905+0000" 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.070438+0000" 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.952 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 367, 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.072942+0000" 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.072955+0000" 2024-09-15T12:24:00.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.073341+0000" 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.073731+0000" 2024-09-15T12:24:00.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.074186+0000" 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.115417+0000" 2024-09-15T12:24:00.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.076743+0000" 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.958 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.077026+0000" 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.960 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.077439+0000" 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.084635+0000" 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.963 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.110245+0000" 2024-09-15T12:24:00.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 373, 2024-09-15T12:24:00.965 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.102283+0000" 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.102301+0000" 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.966 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.102647+0000" 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.102848+0000" 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.105262+0000" 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.968 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.142965+0000" 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.105761+0000" 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.106157+0000" 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.106665+0000" 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.116946+0000" 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.975 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.139958+0000" 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.977 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 398, 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.246658+0000" 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.246674+0000" 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.978 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.246889+0000" 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.247149+0000" 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.248474+0000" 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.294488+0000" 2024-09-15T12:24:00.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.248960+0000" 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.249274+0000" 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.983 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.249851+0000" 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.985 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.256075+0000" 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.290389+0000" 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 405, 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.314234+0000" 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.314249+0000" 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.314823+0000" 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.316728+0000" 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.319385+0000" 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.371951+0000" 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.321700+0000" 2024-09-15T12:24:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.994 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.322126+0000" 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.322877+0000" 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.338469+0000" 2024-09-15T12:24:00.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:00.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.367238+0000" 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 416, 2024-09-15T12:24:01.000 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.382377+0000" 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.382387+0000" 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.382781+0000" 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.382975+0000" 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.383098+0000" 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.440300+0000" 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.004 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.383478+0000" 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.383698+0000" 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.384515+0000" 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.387934+0000" 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.439109+0000" 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.011 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 424, 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.434167+0000" 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.434186+0000" 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.434367+0000" 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.434622+0000" 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.434770+0000" 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.509097+0000" 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.435440+0000" 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.016 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.435905+0000" 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.436533+0000" 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.441100+0000" 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.508392+0000" 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 441, 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557239+0000" 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557247+0000" 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557391+0000" 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557480+0000" 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557619+0000" 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.620004+0000" 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.558262+0000" 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.027 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.558573+0000" 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.559213+0000" 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.562911+0000" 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.615594+0000" 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 446, 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.588494+0000" 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.588517+0000" 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.589222+0000" 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.589407+0000" 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.589773+0000" 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.629166+0000" 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.590413+0000" 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.590738+0000" 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.591355+0000" 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.596638+0000" 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.627202+0000" 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777443, 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4229.0:5084", 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "3.a", 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 30, 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 29, 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "3:57d80787:::benchmark_data_smithi103_30172_object616:head", 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0" 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 406, 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.324030+0000" 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.324048+0000" 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.324292+0000" 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.327212+0000" 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.328647+0000" 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.356119+0000" 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.329422+0000" 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.051 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.329803+0000" 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.330445+0000" 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.334070+0000" 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.056 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.355020+0000" 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 410, 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.370318+0000" 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.370339+0000" 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.371438+0000" 2024-09-15T12:24:01.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.372592+0000" 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.375027+0000" 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.427368+0000" 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.062 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.375721+0000" 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376060+0000" 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376715+0000" 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.067 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.381907+0000" 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.426434+0000" 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.069 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 415, 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380546+0000" 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380558+0000" 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380957+0000" 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.381072+0000" 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.385990+0000" 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.430645+0000" 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.386727+0000" 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.388084+0000" 2024-09-15T12:24:01.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.076 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.388544+0000" 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.398084+0000" 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.429352+0000" 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.081 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 411, 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376161+0000" 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376176+0000" 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376583+0000" 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.376802+0000" 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.377914+0000" 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.447684+0000" 2024-09-15T12:24:01.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378703+0000" 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378937+0000" 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.091 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.379625+0000" 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.387597+0000" 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.446757+0000" 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.095 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 418, 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.387024+0000" 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.387035+0000" 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.387291+0000" 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.389746+0000" 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.396660+0000" 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.479986+0000" 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.398815+0000" 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.400766+0000" 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.406324+0000" 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.415736+0000" 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.478934+0000" 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 428, 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458251+0000" 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458259+0000" 2024-09-15T12:24:01.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458375+0000" 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458498+0000" 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.109 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.461015+0000" 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.493106+0000" 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.461674+0000" 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.462047+0000" 2024-09-15T12:24:01.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.113 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.464473+0000" 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.115 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.472281+0000" 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.116 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.492128+0000" 2024-09-15T12:24:01.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 419, 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.393307+0000" 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.393316+0000" 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.393477+0000" 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.119 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.395206+0000" 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.120 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.398048+0000" 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.503299+0000" 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.404387+0000" 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.408856+0000" 2024-09-15T12:24:01.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.125 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.409749+0000" 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.419572+0000" 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.502150+0000" 2024-09-15T12:24:01.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 421, 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.401611+0000" 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.401620+0000" 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.404497+0000" 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.404918+0000" 2024-09-15T12:24:01.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.405735+0000" 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.513791+0000" 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.407799+0000" 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.410081+0000" 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.411029+0000" 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.417518+0000" 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.429405+0000" 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.512552+0000" 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 427, 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.457605+0000" 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.140 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.457617+0000" 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.457751+0000" 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.457880+0000" 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458143+0000" 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.551890+0000" 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458792+0000" 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.459122+0000" 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.459812+0000" 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.463712+0000" 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.149 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.550475+0000" 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 433, 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.482456+0000" 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.482464+0000" 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.151 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.482623+0000" 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.152 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.482720+0000" 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.484712+0000" 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574361+0000" 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.485612+0000" 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.155 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.486245+0000" 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.490184+0000" 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.494824+0000" 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.566887+0000" 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 437, 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.544343+0000" 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.544358+0000" 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.544549+0000" 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.544649+0000" 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.546494+0000" 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.594883+0000" 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.547253+0000" 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.166 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.547607+0000" 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.548400+0000" 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.553379+0000" 2024-09-15T12:24:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.593700+0000" 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.173 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 438, 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.552588+0000" 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.552606+0000" 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.552739+0000" 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.552873+0000" 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.558840+0000" 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.609951+0000" 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.559927+0000" 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.560567+0000" 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.180 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.561194+0000" 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.568410+0000" 2024-09-15T12:24:01.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.608984+0000" 2024-09-15T12:24:01.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 442, 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.571681+0000" 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.571709+0000" 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.571919+0000" 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.572095+0000" 2024-09-15T12:24:01.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574875+0000" 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.622843+0000" 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.575878+0000" 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.576378+0000" 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.190 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.577452+0000" 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.580878+0000" 2024-09-15T12:24:01.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.621788+0000" 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 447, 2024-09-15T12:24:01.195 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.625766+0000" 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.625785+0000" 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.626430+0000" 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.197 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.627616+0000" 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.629983+0000" 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.662972+0000" 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.630688+0000" 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.200 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.631020+0000" 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.631643+0000" 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.203 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.633528+0000" 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.662090+0000" 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 450, 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.637654+0000" 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.637683+0000" 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.637957+0000" 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.638468+0000" 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.639835+0000" 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.679676+0000" 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.640541+0000" 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.640871+0000" 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.641546+0000" 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.214 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.215 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.644268+0000" 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.672422+0000" 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 455, 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.661241+0000" 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.661273+0000" 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.661480+0000" 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.661676+0000" 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.663406+0000" 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.220 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.691914+0000" 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.664117+0000" 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.664657+0000" 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.224 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.665328+0000" 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.668159+0000" 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.227 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.689311+0000" 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 456, 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.665922+0000" 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.665931+0000" 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.666140+0000" 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.666247+0000" 2024-09-15T12:24:01.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.669091+0000" 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.709313+0000" 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.670363+0000" 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.672170+0000" 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.234 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.677970+0000" 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.237 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.691012+0000" 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.708329+0000" 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 458, 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.670483+0000" 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.670497+0000" 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.670778+0000" 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.241 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.670918+0000" 2024-09-15T12:24:01.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.677360+0000" 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.243 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.724045+0000" 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.679295+0000" 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.245 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.680091+0000" 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.682612+0000" 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.698359+0000" 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.723173+0000" 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 457, 2024-09-15T12:24:01.251 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.668244+0000" 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.668264+0000" 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.668520+0000" 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.253 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.668665+0000" 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.669214+0000" 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.726163+0000" 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.255 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.677246+0000" 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.678733+0000" 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.682827+0000" 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.697191+0000" 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.725008+0000" 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.262 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 461, 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.742331+0000" 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.742344+0000" 2024-09-15T12:24:01.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.742499+0000" 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.264 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.742634+0000" 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.746796+0000" 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.747398+0000" 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.267 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.747708+0000" 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.748317+0000" 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.753077+0000" 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.272 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 463, 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.748894+0000" 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.748906+0000" 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.275 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.749061+0000" 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.749177+0000" 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.749881+0000" 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.278 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.750541+0000" 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.750806+0000" 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.281 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.751324+0000" 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.284 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.285 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 464, 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.749591+0000" 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.749603+0000" 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.750249+0000" 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.750474+0000" 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.757539+0000", 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.289 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.292 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.296 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.297 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 408, 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.331634+0000" 2024-09-15T12:24:01.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.331660+0000" 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.334504+0000" 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.300 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.337400+0000" 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.348299+0000" 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.302 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.383450+0000" 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.348767+0000" 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.348965+0000" 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.306 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.349349+0000" 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.351071+0000" 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.382670+0000" 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.310 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 409, 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.350212+0000" 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.350230+0000" 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.353976+0000" 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.312 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.358530+0000" 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.361959+0000" 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.393248+0000" 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.362379+0000" 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.362591+0000" 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.362983+0000" 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.318 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.319 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.365328+0000" 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.392235+0000" 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.321 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 413, 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378607+0000" 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.322 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378615+0000" 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378809+0000" 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378945+0000" 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.324 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.379278+0000" 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.419963+0000" 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.379776+0000" 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380062+0000" 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380502+0000" 2024-09-15T12:24:01.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.384383+0000" 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.331 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.418658+0000" 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 412, 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.377567+0000" 2024-09-15T12:24:01.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.377578+0000" 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.377950+0000" 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378183+0000" 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.378505+0000" 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.445303+0000" 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.379157+0000" 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.379890+0000" 2024-09-15T12:24:01.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.380888+0000" 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.385386+0000" 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.444388+0000" 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.343 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 417, 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.344 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.384654+0000" 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.384662+0000" 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.385478+0000" 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.345 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.385844+0000" 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.390660+0000" 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.454469+0000" 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.391858+0000" 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.348 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.392430+0000" 2024-09-15T12:24:01.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.393416+0000" 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.396986+0000" 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.453838+0000" 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 422, 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.407790+0000" 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.407809+0000" 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.410242+0000" 2024-09-15T12:24:01.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.357 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.410525+0000" 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.412829+0000" 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458061+0000" 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.413256+0000" 2024-09-15T12:24:01.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.360 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.413454+0000" 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.414000+0000" 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.364 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.415718+0000" 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.457476+0000" 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 425, 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.435777+0000" 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.435787+0000" 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.436029+0000" 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.436517+0000" 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.369 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.437689+0000" 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.463141+0000" 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.438118+0000" 2024-09-15T12:24:01.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.372 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.438305+0000" 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.438693+0000" 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.440469+0000" 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.462425+0000" 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 423, 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.419682+0000" 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.419699+0000" 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.419941+0000" 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.380 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.420200+0000" 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.420421+0000" 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.485838+0000" 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.420936+0000" 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.421139+0000" 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.385 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.421602+0000" 2024-09-15T12:24:01.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.423573+0000" 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.388 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.485050+0000" 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 426, 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.390 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.445606+0000" 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.445618+0000" 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.445790+0000" 2024-09-15T12:24:01.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.392 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.445902+0000" 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.446169+0000" 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.523668+0000" 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.394 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.446806+0000" 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.447164+0000" 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.397 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.447873+0000" 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.399 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.449986+0000" 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.400 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.522894+0000" 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 429, 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.402 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458951+0000" 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.458958+0000" 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.459092+0000" 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.459179+0000" 2024-09-15T12:24:01.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.459423+0000" 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.405 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.543416+0000" 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.460071+0000" 2024-09-15T12:24:01.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.460374+0000" 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.408 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.460981+0000" 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.464079+0000" 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.542726+0000" 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 435, 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.532917+0000" 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.532931+0000" 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.533108+0000" 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.533211+0000" 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.533552+0000" 2024-09-15T12:24:01.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.560223+0000" 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.534060+0000" 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.534276+0000" 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.534774+0000" 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.422 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.536859+0000" 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.558889+0000" 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 440, 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.556333+0000" 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.556342+0000" 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.556576+0000" 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.426 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.556684+0000" 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.427 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.557743+0000" 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.587345+0000" 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.558418+0000" 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.559083+0000" 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.559889+0000" 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.562548+0000" 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.586419+0000" 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.435 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 434, 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.504290+0000" 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.504303+0000" 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.504459+0000" 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.504568+0000" 2024-09-15T12:24:01.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.504806+0000" 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.593207+0000" 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.505199+0000" 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.505388+0000" 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.441 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.505797+0000" 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.444 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.507846+0000" 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.592481+0000" 2024-09-15T12:24:01.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 444, 2024-09-15T12:24:01.447 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.580261+0000" 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.580274+0000" 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.448 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.580443+0000" 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.580546+0000" 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.583117+0000" 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.625117+0000" 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.583778+0000" 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.584089+0000" 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.454 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.584729+0000" 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.588377+0000" 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.457 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.624180+0000" 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 443, 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574255+0000" 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574270+0000" 2024-09-15T12:24:01.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574407+0000" 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574542+0000" 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.574915+0000" 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.630207+0000" 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.575575+0000" 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.575905+0000" 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.576314+0000" 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.467 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.578763+0000" 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.629332+0000" 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 449, 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.633691+0000" 2024-09-15T12:24:01.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.633707+0000" 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.633952+0000" 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.472 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.634698+0000" 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.635072+0000" 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.690196+0000" 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.635467+0000" 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.635681+0000" 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.636067+0000" 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.478 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.638610+0000" 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.479 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.689296+0000" 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 452, 2024-09-15T12:24:01.481 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.641727+0000" 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.641743+0000" 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.641965+0000" 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.642170+0000" 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.643403+0000" 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.712333+0000" 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.485 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.643809+0000" 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.644014+0000" 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.487 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.644578+0000" 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.646598+0000" 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.711709+0000" 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 453, 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.646975+0000" 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.493 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.646985+0000" 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.496 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.647113+0000" 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.647241+0000" 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.650952+0000" 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.498 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.714505+0000" 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.651322+0000" 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.500 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.651529+0000" 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.651923+0000" 2024-09-15T12:24:01.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.653861+0000" 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.505 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.713817+0000" 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 460, 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.691865+0000" 2024-09-15T12:24:01.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.691886+0000" 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.692070+0000" 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:01.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.692416+0000" 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.692711+0000" 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.510 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.722750+0000" 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:01.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.693161+0000" 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.693357+0000" 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.514 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.693756+0000" 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.695944+0000" 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.516 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:01.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:00.722121+0000" 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:24:01.518 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:24:02.089 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:02.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:02.249+0000 7f8ffdefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:02.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:02.256+0000 7f8ffdefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:02.632 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:02.632 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:04.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:04.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:04.793+0000 7f9aaa991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:04.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:04.803+0000 7f9aaa991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:05.182 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:05.186 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:05.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:06.520 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:24:06.521 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:06.521 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-15T12:24:06.521 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:24:06.521 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:24:06.523 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:24:06.524 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:24:06.660 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:24:06.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:24:06.663 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:24:07.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:07.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:07.338+0000 7f19b52ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:07.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:07.346+0000 7f19b52ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:07.969 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:07.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:07.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:09.974 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:10.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:10.116+0000 7f4bab4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:10.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:10.125+0000 7f4bab4ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:10.432 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:10.436 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:10.437 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:11.664 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:24:11.817 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:24:11.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:24:11.821 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:24:12.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:12.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:12.610+0000 7fcad0e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:12.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:12.619+0000 7fcad0e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:13.058 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:13.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:13.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:15.063 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:15.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:15.221+0000 7f6494efd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:15.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:15.230+0000 7f6494efd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:15.638 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:15.642 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:15.643 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:15.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Removed 4465 objects 2024-09-15T12:24:15.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Clean up completed and total clean up time :17.3521 2024-09-15T12:24:16.347 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-15T12:24:16.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-15T12:24:16.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:16.452+0000 7f0e032d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:16.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:16.465+0000 7f0e032d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:16.666 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_0' removed 2024-09-15T12:24:16.686 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-15T12:24:16.687 INFO:tasks.radosbench:Beginning radosbench... 2024-09-15T12:24:16.687 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-15T12:24:16.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_1 16 2024-09-15T12:24:16.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:24:16.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:16.831+0000 7f8cc72bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:16.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:16.847+0000 7f8cc72bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:16.934 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:24:16.937 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:24:16.938 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:24:17.643 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:17.667 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_1' created 2024-09-15T12:24:17.687 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-15T12:24:17.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:17.793+0000 7fc96b488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:17.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:17.801+0000 7fc96b488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:17.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:17.875+0000 7f33a4791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:17.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:17.889+0000 7f33a4791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:17.892 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:17.896 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:17.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:18.676 INFO:teuthology.orchestra.run.smithi103.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-15T12:24:19.698 DEBUG:teuthology.orchestra.run.smithi103:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-15T12:24:19.700 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-15T12:24:19.701 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-15T12:24:19.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:hints = 1 2024-09-15T12:24:19.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-15T12:24:19.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object prefix: benchmark_data_smithi103_32472 2024-09-15T12:24:19.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:24:19.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 0 0 0 0 0 0 - 0 2024-09-15T12:24:19.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:20.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:20.056+0000 7f2c38791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:20.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:20.071+0000 7f2c38791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:20.415 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:24:20.418 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:20.419 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:20.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1 16 214 198 12.3743 12.375 0.122272 0.0741231 2024-09-15T12:24:21.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2 16 215 199 6.21828 0.0625 0.152373 0.0745164 2024-09-15T12:24:21.939 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:24:22.130 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:24:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [ 2024-09-15T12:24:22.130 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 96, 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "2.6", 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 159.932052612s int 16.000000000s) +create_info", 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.131 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.731471+0000" 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.731476+0000" 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.919354+0000" 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.132 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.924818+0000" 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.959092+0000" 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989591+0000" 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.990214+0000" 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.137 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 97, 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.f", 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 2 log log((0'0,34'13], crt=34'13) pi ([32,34] all_participants={0, 2} intervals=[([32,34] acting={0, 2})]) pg_lease(ru 0.000000000s ub 159.937866211s int 16.000000000s) +create_info", 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.734930+0000" 2024-09-15T12:24:22.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.734946+0000" 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.924300+0000" 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.961353+0000" 2024-09-15T12:24:22.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.990106+0000" 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.990260+0000" 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.074110+0000" 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.074228+0000", 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.143 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 125 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.144 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 103, 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "1.0", 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 2 info: 1.0( v 11'52 (0'0,11'52] local-lis/les=35/36 n=2 ec=10/10 lis/c=35/23 les/c/f=36/24/0 sis=35 pruub=14.035268784s) pg_lease_ack(ruub 29.530551910s)", 2024-09-15T12:24:22.145 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.828548+0000" 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.828564+0000" 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.991123+0000" 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.995715+0000" 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.998507+0000" 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.148 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.074601+0000" 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.150 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.074946+0000" 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.151 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 118, 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.4", 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 2 info: 4.4( v 34'12 (0'0,34'12] local-lis/les=35/36 n=12 ec=32/32 lis/c=35/0 les/c/f=36/0/0 sis=35) pg_lease(ru 159.929336548s ub 160.225585938s int 16.000000000s)", 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.152 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.023233+0000" 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.023247+0000" 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.074909+0000" 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.154 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.085432+0000" 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.089746+0000" 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.091727+0000" 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.157 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.095515+0000" 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.158 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 120, 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.4", 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 RequestRecoveryPrio: priority 190", 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.159 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.033630+0000" 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.033646+0000" 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.091835+0000" 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.160 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.161 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.097129+0000", 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::get_pg_mapping", 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.162 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.163 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.164 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.165 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 121, 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.7", 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 2 info: 4.7( v 34'13 (0'0,34'13] local-lis/les=35/36 n=13 ec=32/32 lis/c=35/0 les/c/f=36/0/0 sis=35) pg_lease(ru 159.933517456s ub 160.262908936s int 16.000000000s)", 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.166 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.048944+0000" 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.048959+0000" 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.167 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.168 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.169 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.170 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.172 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 122, 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.7", 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MLease epoch 36 from osd.2 pg_lease(ru 159.933517456s ub 160.262908936s int 16.000000000s)", 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.049862+0000" 2024-09-15T12:24:22.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.049872+0000", 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::await_active", 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 3, 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 121 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.174 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.179 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 124, 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.7", 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.180 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 RequestRecoveryPrio: priority 190", 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.064788+0000" 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.064799+0000", 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::await_active", 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 3, 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 121 2024-09-15T12:24:22.181 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.182 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 126, 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "3", 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "1.0", 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.187 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 AllReplicasRecovered", 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.089595+0000" 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.091886+0000", 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.189 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 103 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.190 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 125, 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.f", 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.191 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 119, 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 123, 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.192 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777274, 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.0", 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 0 log log((0'0,34'11], crt=34'11) pi ([32,34] all_participants={0, 1} intervals=[([32,34] acting={0, 1})]) pg_lease(ru 0.000000000s ub 160.020538330s int 16.000000000s) +create_info", 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.720538+0000" 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.193 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.720547+0000" 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.908456+0000" 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.908642+0000" 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.195 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.908930+0000" 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.909172+0000" 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.196 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988242+0000" 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988303+0000", 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 16777288 2024-09-15T12:24:22.198 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554491, 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.9", 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 1 log log((0'0,34'12], crt=34'12) pi ([32,34] all_participants={0, 1} intervals=[([32,34] acting={0, 1})]) pg_lease(ru 0.000000000s ub 51.248153687s int 16.000000000s) +create_info", 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.200 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.737451+0000" 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.737461+0000" 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.912766+0000" 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.912928+0000" 2024-09-15T12:24:22.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.913635+0000" 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.018452+0000" 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.045228+0000" 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.056006+0000" 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.205 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 94, 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.206 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.e", 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 2 log log((0'0,34'17], crt=34'17) pi ([32,34] all_participants={1, 2} intervals=[([32,34] acting={1, 2})]) pg_lease(ru 0.000000000s ub 159.936935425s int 16.000000000s) +create_info", 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.728020+0000" 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.728034+0000" 2024-09-15T12:24:22.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.912250+0000" 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.208 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.914416+0000" 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.209 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.919455+0000" 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.056043+0000" 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.087496+0000" 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.211 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.087648+0000", 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 16777296 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.212 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.213 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777294, 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "3", 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "3.6", 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 DeleteSome", 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.061173+0000" 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.215 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.061614+0000" 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.216 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777295, 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "3", 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "3.e", 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 35, 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 35 epoch_requested: 35 DeleteReserved", 2024-09-15T12:24:22.217 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.061534+0000" 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.062153+0000" 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.219 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 105, 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.220 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "2.5", 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 2 info: 2.5( empty local-lis/les=35/36 n=0 ec=12/12 lis/c=35/23 les/c/f=36/24/0 sis=35 pruub=14.027475357s) pg_lease_ack(ruub 29.542703629s)", 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.839611+0000" 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.839618+0000" 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.221 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.995889+0000" 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.222 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.998568+0000" 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.223 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.075226+0000" 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.224 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.087833+0000" 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.087883+0000" 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.226 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554495, 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.5", 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.227 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 1 log log((0'0,34'14], crt=34'14) pi ([32,34] all_participants={0, 1} intervals=[([32,34] acting={0, 1})]) pg_lease(ru 0.000000000s ub 51.283477783s int 16.000000000s) +create_info", 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.764443+0000" 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.764449+0000" 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.228 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988783+0000" 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989697+0000" 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.077023+0000" 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.087908+0000", 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PGCreation", 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.5", 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: "creating": true 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.231 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.232 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.233 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554496, 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.234 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "2.1", 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 1 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 51.280094147s int 16.000000000s) +create_info", 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.765011+0000" 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.765017+0000" 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.235 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989786+0000" 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076376+0000" 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.237 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.077067+0000", 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554495 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.238 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.239 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.240 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777297, 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.241 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "3", 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "2.5", 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 AllReplicasRecovered", 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.092963+0000" 2024-09-15T12:24:22.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.093013+0000", 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 105 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.244 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777288, 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.0", 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.245 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777296, 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.e", 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.246 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554493, 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.8", 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 1 log log((0'0,34'13], crt=34'13) pi ([32,34] all_participants={0, 1} intervals=[([32,34] acting={0, 1})]) pg_lease(ru 0.000000000s ub 51.259544373s int 16.000000000s) +create_info", 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.247 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.754574+0000" 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.754585+0000" 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.248 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.913306+0000" 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.913512+0000" 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988548+0000" 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989439+0000" 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.075983+0000" 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076046+0000", 2024-09-15T12:24:22.252 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554518 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.253 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554494, 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.a", 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.254 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 1 log log((0'0,34'9], crt=34'9) pi ([32,34] all_participants={1, 2} intervals=[([32,34] acting={1, 2})]) pg_lease(ru 0.000000000s ub 51.270839691s int 16.000000000s) +create_info", 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.763712+0000" 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.763719+0000" 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.255 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.913554+0000" 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.256 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988695+0000" 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989613+0000" 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.257 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076114+0000" 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.104865+0000" 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.259 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.104971+0000", 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554520 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.260 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554505, 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.261 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.c", 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 1 info: 4.c( v 34'17 (0'0,34'17] local-lis/les=35/36 n=17 ec=32/32 lis/c=35/0 les/c/f=36/0/0 sis=35) pg_lease_ack(ruub 29.540771484s)", 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.832610+0000" 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.832625+0000" 2024-09-15T12:24:22.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076451+0000" 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076534+0000" 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.264 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.076767+0000" 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.265 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.107057+0000" 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.266 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.110375+0000", 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 16777286 2024-09-15T12:24:22.267 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.268 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777271, 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.6", 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 0 log log((0'0,34'13], crt=34'13) pi ([32,34] all_participants={0, 2} intervals=[([32,34] acting={0, 2})]) pg_lease(ru 0.000000000s ub 160.013092041s int 16.000000000s) +create_info", 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.269 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.716486+0000" 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.716495+0000" 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.716757+0000" 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.270 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.907834+0000" 2024-09-15T12:24:22.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.908259+0000" 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.272 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.909137+0000" 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988242+0000" 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.988440+0000", 2024-09-15T12:24:22.274 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PeeringEvent::PGPipeline::process", 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554508 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.275 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 16777286, 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.c", 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.276 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 0 info: 4.c( v 34'17 lc 34'15 (0'0,34'17] local-lis/les=35/36 n=17 ec=32/32 lis/c=35/0 les/c/f=36/0/0 sis=35) pg_lease_ack(ruub 29.540771484s)", 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.836215+0000" 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.836221+0000" 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.910058+0000" 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.910211+0000" 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.990469+0000" 2024-09-15T12:24:22.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.105126+0000" 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.281 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.105163+0000" 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.282 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 98, 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2", 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "2.4", 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 35 MLogRec from 2 log log((0'0,0'0], crt=0'0) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 159.929733276s int 16.000000000s) +create_info", 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.283 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.737953+0000" 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.737965+0000" 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.961729+0000" 2024-09-15T12:24:22.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.989975+0000" 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:21.996136+0000" 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.286 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.106841+0000" 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.287 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.106949+0000" 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.288 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.289 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554513, 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.2", 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 RequestRecoveryPrio: priority 190", 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.043566+0000" 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.290 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.043580+0000" 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.292 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.294 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.295 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554514, 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.296 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.1", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MLease epoch 36 from osd.1 pg_lease(ru 51.242401123s ub 51.574138641s int 16.000000000s)", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.054454+0000" 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.054461+0000", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::await_active", 2024-09-15T12:24:22.297 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 4, 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554513 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.298 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.299 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.300 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554515, 2024-09-15T12:24:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.1", 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 MInfoRec from 1 info: 4.1( v 34'12 (0'0,34'12] local-lis/les=35/36 n=12 ec=32/32 lis/c=35/0 les/c/f=36/0/0 sis=35) pg_lease(ru 51.242401123s ub 51.574138641s int 16.000000000s)", 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.055082+0000" 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.055094+0000", 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::await_active", 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 4, 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554513 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.305 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.306 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.307 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.308 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.309 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.310 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554516, 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "3", 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "3.1", 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 35, 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 35, 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 35 epoch_requested: 35 DeleteReserved", 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.065247+0000" 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.312 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.065403+0000" 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.313 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554519, 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.1", 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 36, 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 36, 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 36 epoch_requested: 36 RequestRecoveryPrio: priority 190", 2024-09-15T12:24:22.314 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.070136+0000" 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:24:22.070147+0000", 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "ConnectionPipeline::await_active", 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 4, 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 33554513 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.315 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.316 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.317 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554508, 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.6", 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.321 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554518, 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.8", 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.322 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "pg_advance_map", 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554520, 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "PGAdvanceMap": { 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.a", 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "from": 35, 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "to": 36, 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: "do_init": true 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:24:22.323 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 33554517, 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:24:22.324 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:24:22.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:22.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:22.557+0000 7f6952791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:22.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:22.566+0000 7f6952791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:22.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3 16 269 253 5.27042 3.375 0.0571122 0.16538 2024-09-15T12:24:23.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 4 16 441 425 6.64008 10.75 0.0440739 0.146638 2024-09-15T12:24:24.769 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 5 16 553 537 6.71194 7 0.375167 0.14244 2024-09-15T12:24:24.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:25.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:25.081+0000 7fe83b674640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:25.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:25.089+0000 7fe83b674640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:25.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 6 16 732 716 7.4577 11.1875 0.0433215 0.132819 2024-09-15T12:24:26.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 7 16 956 940 8.39215 14 0.0661063 0.118446 2024-09-15T12:24:27.326 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:24:27.326 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:27.326 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 2, 1, 3] 2024-09-15T12:24:27.327 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T12:24:27.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:24:27.410 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T12:24:27.466 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:27.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:27.568+0000 7f443eaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:27.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:27.573+0000 7f443eaec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:27.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 8 16 1088 1072 8.37423 8.25 0.0151862 0.11289 2024-09-15T12:24:28.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 9 16 1088 1072 7.44373 0 - 0.11289 2024-09-15T12:24:29.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 10 16 1088 1072 6.69933 0 - 0.11289 2024-09-15T12:24:30.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 11 16 1088 1072 6.09027 0 - 0.11289 2024-09-15T12:24:31.770 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 12 16 1088 1072 5.58273 0 - 0.11289 2024-09-15T12:24:32.411 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 1, 3] 2024-09-15T12:24:32.411 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:32.411 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-15T12:24:32.411 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T12:24:32.411 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-15T12:24:32.414 INFO:tasks.ceph.osd.0:Started 2024-09-15T12:24:32.415 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:24:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:24:32.529 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:24:32.530 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:24:32.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 13 16 1088 1072 5.15329 0 - 0.11289 2024-09-15T12:24:33.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 14 16 1088 1072 4.78519 0 - 0.11289 2024-09-15T12:24:34.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 15 16 1088 1072 4.46618 0 - 0.11289 2024-09-15T12:24:35.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 16 16 1088 1072 4.18705 0 - 0.11289 2024-09-15T12:24:36.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 17 16 1088 1072 3.94075 0 - 0.11289 2024-09-15T12:24:37.531 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:24:37.631 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:24:37.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:24:37.635 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:24:37.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 18 16 1088 1072 3.72181 0 - 0.11289 2024-09-15T12:24:38.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 19 16 1088 1072 3.52593 0 - 0.11289 2024-09-15T12:24:39.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:24:39.771072+0000 min lat: 0.0148315 max lat: 2.12285 avg lat: 0.11289 2024-09-15T12:24:39.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:24:39.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 20 16 1088 1072 3.34963 0 - 0.11289 2024-09-15T12:24:40.103 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:24:40.274 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:24:40.274 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:24:40.771 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 21 16 1088 1072 3.19012 0 - 0.11289 2024-09-15T12:24:41.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 22 16 1088 1072 3.04511 0 - 0.11289 2024-09-15T12:24:42.275 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:42.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:42.402+0000 7f043b4c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:42.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:42.410+0000 7f043b4c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:42.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:24:42.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 23 16 1147 1131 3.07302 0.245833 0.046813 0.324697 2024-09-15T12:24:42.848 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:24:42.848 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 247, 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4363.0:1150", 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "4.f", 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 39, 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 38, 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "4:f15e3ef3:::benchmark_data_smithi103_32472_object1149:head", 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "2" 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.849 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 248, 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.4363.0:1147", 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "4.1", 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 39, 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 38, 2024-09-15T12:24:42.850 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "4:85c06d9b:::benchmark_data_smithi103_32472_object1146:head", 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "1" 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777294, 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.797789+0000" 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.797802+0000" 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.797971+0000" 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.852 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.798129+0000" 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.798268+0000" 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.798686+0000" 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.855 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.798901+0000" 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.856 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.858 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.799306+0000" 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.802592+0000" 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.862 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777295, 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800050+0000" 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800061+0000" 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.863 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800192+0000" 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.864 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800319+0000" 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.865 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800417+0000" 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.866 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.800836+0000" 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.801008+0000" 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.868 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.801475+0000" 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.870 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.805436+0000" 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777277, 2024-09-15T12:24:42.874 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.576439+0000" 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.576484+0000" 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.875 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.576793+0000" 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.876 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.576953+0000" 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.577161+0000" 2024-09-15T12:24:42.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.580069+0000" 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.577840+0000" 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.578226+0000" 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.578914+0000" 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.882 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.883 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.884 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.885 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777278, 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.580459+0000" 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.580478+0000" 2024-09-15T12:24:42.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.580732+0000" 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.580968+0000" 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.888 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.581273+0000" 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.583871+0000" 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.581924+0000" 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.582274+0000" 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.893 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.583029+0000" 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.894 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.895 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.896 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.897 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777279, 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.584216+0000" 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.898 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.584231+0000" 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.584462+0000" 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.900 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.584637+0000" 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.584892+0000" 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.901 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.587728+0000" 2024-09-15T12:24:42.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.585521+0000" 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.903 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.586013+0000" 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.904 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.905 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.586721+0000" 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.909 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.910 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777280, 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.911 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.588175+0000" 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.588191+0000" 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.588415+0000" 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.912 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.588586+0000" 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.588796+0000" 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.914 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.669158+0000" 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.589441+0000" 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.589806+0000" 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.917 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.918 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.590718+0000" 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.919 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.920 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.595661+0000" 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.921 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.922 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.668044+0000" 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.923 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777282, 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594215+0000" 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594224+0000" 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594383+0000" 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.925 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594486+0000" 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.926 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594757+0000" 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.673090+0000" 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.596348+0000" 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.928 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.597888+0000" 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.930 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.600468+0000" 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.932 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.607765+0000" 2024-09-15T12:24:42.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.671248+0000" 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777284, 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.604965+0000" 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.604977+0000" 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.605132+0000" 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.937 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.605295+0000" 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.938 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.605486+0000" 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.939 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.677058+0000" 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.605875+0000" 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.606069+0000" 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.941 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.606530+0000" 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.943 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.612813+0000" 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.945 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.675988+0000" 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777287, 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.632536+0000" 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.947 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.632564+0000" 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.632802+0000" 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.948 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.632971+0000" 2024-09-15T12:24:42.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.633127+0000" 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.692105+0000" 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.633744+0000" 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.951 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.634063+0000" 2024-09-15T12:24:42.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.953 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.634699+0000" 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.956 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.637230+0000" 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.690917+0000" 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777288, 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.648457+0000" 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.648477+0000" 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.959 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.648816+0000" 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.960 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.649036+0000" 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.649204+0000" 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.961 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.715835+0000" 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.649849+0000" 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.650174+0000" 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.965 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.650839+0000" 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.967 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.652987+0000" 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.968 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.715142+0000" 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777289, 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.970 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.671416+0000" 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.671432+0000" 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.672339+0000" 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.972 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.672522+0000" 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.672690+0000" 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.974 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.734725+0000" 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.673818+0000" 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.674116+0000" 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.978 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.674548+0000" 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.677758+0000" 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.733611+0000" 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.982 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777290, 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.697499+0000" 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.697526+0000" 2024-09-15T12:24:42.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.697658+0000" 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.697793+0000" 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.986 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.697898+0000" 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.746732+0000" 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.698380+0000" 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.989 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.698642+0000" 2024-09-15T12:24:42.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:42.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.993 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:42.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.699070+0000" 2024-09-15T12:24:42.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.997 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.700389+0000" 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.998 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.745633+0000" 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:42.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777291, 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.716805+0000" 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.716821+0000" 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.716977+0000" 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.717074+0000" 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.717174+0000" 2024-09-15T12:24:43.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.756882+0000" 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.717587+0000" 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.005 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.717778+0000" 2024-09-15T12:24:43.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.718160+0000" 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.719706+0000" 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.753574+0000" 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777293, 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.747978+0000" 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.748001+0000" 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.748220+0000" 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.748409+0000" 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.748579+0000" 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.797077+0000" 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.749154+0000" 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.749460+0000" 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.749941+0000" 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.752120+0000" 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.796155+0000" 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777296, 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.830745+0000" 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.830757+0000" 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.830938+0000" 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.831112+0000" 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.831764+0000" 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.832167+0000" 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.031 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.832459+0000" 2024-09-15T12:24:43.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.833264+0000" 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777297, 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.831819+0000" 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.039 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.831832+0000" 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.832055+0000" 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.832194+0000" 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.833049+0000" 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.834266+0000" 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.835326+0000" 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.835908+0000" 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.050 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777281, 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.593020+0000" 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.593037+0000" 2024-09-15T12:24:43.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.593201+0000" 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.593456+0000" 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.053 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594193+0000" 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.669453+0000" 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594617+0000" 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.055 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.594819+0000" 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.058 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.595211+0000" 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.596676+0000" 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.060 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.061 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.668579+0000" 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.062 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777283, 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.603914+0000" 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.603937+0000" 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.604084+0000" 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.064 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.604240+0000" 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.065 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.605307+0000" 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.066 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.673187+0000" 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.606000+0000" 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.606307+0000" 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.069 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.606939+0000" 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.072 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.609058+0000" 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.672617+0000" 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.075 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777285, 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.609370+0000" 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.076 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.609381+0000" 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.610926+0000" 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.078 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.611158+0000" 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.612726+0000" 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.080 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.682881+0000" 2024-09-15T12:24:43.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.613563+0000" 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.613900+0000" 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.085 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.614570+0000" 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.087 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.619937+0000" 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.088 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.679819+0000" 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777286, 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.611932+0000" 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.611945+0000" 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.612790+0000" 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.613555+0000" 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.093 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.615895+0000" 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.094 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.685771+0000" 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.616590+0000" 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.616856+0000" 2024-09-15T12:24:43.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.097 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.098 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.617268+0000" 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.621937+0000" 2024-09-15T12:24:43.100 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.101 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.684835+0000" 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.102 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16777292, 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.723672+0000" 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.723683+0000" 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.103 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.723833+0000" 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:24:43.104 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.723934+0000" 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.105 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.724227+0000" 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.106 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.769143+0000" 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.724905+0000" 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.725235+0000" 2024-09-15T12:24:43.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.725899+0000" 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.112 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.728668+0000" 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.113 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:24:42.768539+0000" 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:24:43.114 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:24:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:24:43.115 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:24:43.115 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:24:43.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 24 16 1359 1343 3.49701 13.25 0.0410501 0.285477 2024-09-15T12:24:44.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:44.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:44.732+0000 7f21f3d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:44.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:44.741+0000 7f21f3d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:44.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 25 16 1579 1563 3.90707 13.75 0.0573015 0.255562 2024-09-15T12:24:45.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 26 16 1778 1762 4.23512 12.4375 0.0762747 0.235621 2024-09-15T12:24:46.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 27 16 1980 1964 4.54581 12.625 0.021145 0.21964 2024-09-15T12:24:47.044 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:47.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:47.193+0000 7fe7a8f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:47.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:47.202+0000 7fe7a8f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:47.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 28 16 2194 2178 4.86109 13.375 0.0411032 0.205476 2024-09-15T12:24:48.117 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 3, 0] 2024-09-15T12:24:48.117 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:48.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:24:48.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:48.255+0000 7fdf29391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:48.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:48.268+0000 7fdf29391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:48.604 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:24:48.604 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:24:48.621 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-15T12:24:48.037704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48103987,"num_objects":728,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":804,"num_write_kb":47508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":8323072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":780,"ondisk_log_size":780,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":163372,"kb_used_data":17884,"kb_used_omap":0,"kb_used_meta":145408,"kb_avail":377323988,"statfs":{"total":386547056640,"available":386379763712,"internally_reserved":0,"allocated":18313216,"data_stored":276181633,"data_compressed":13102643,"data_compressed_allocated":17252352,"data_compressed_original":275791872,"omap_allocated":0,"internal_metadata":148897792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.938370"},"pg_stats":[{"pgid":"4.f","version":"36'81","reported_seq":127,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.273292+0000","last_change":"2024-09-15T12:24:25.203380+0000","last_active":"2024-09-15T12:24:27.273292+0000","last_peered":"2024-09-15T12:24:27.273292+0000","last_clean":"2024-09-15T12:24:27.273292+0000","last_became_active":"2024-09-15T12:24:22.178371+0000","last_became_peered":"2024-09-15T12:24:22.178371+0000","last_unstale":"2024-09-15T12:24:27.273292+0000","last_undegraded":"2024-09-15T12:24:27.273292+0000","last_fullsized":"2024-09-15T12:24:27.273292+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5308416,"num_objects":81,"num_object_clones":0,"num_object_copies":243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":5184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.e","version":"36'66","reported_seq":120,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.228153+0000","last_change":"2024-09-15T12:24:25.149438+0000","last_active":"2024-09-15T12:24:27.228153+0000","last_peered":"2024-09-15T12:24:27.228153+0000","last_clean":"2024-09-15T12:24:27.228153+0000","last_became_active":"2024-09-15T12:24:22.192218+0000","last_became_peered":"2024-09-15T12:24:22.192218+0000","last_unstale":"2024-09-15T12:24:27.228153+0000","last_undegraded":"2024-09-15T12:24:27.228153+0000","last_fullsized":"2024-09-15T12:24:27.228153+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"36'69","reported_seq":87,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.336365+0000","last_change":"2024-09-15T12:24:18.848023+0000","last_active":"2024-09-15T12:24:27.336365+0000","last_peered":"2024-09-15T12:24:27.336365+0000","last_clean":"2024-09-15T12:24:27.336365+0000","last_became_active":"2024-09-15T12:24:18.841704+0000","last_became_peered":"2024-09-15T12:24:18.841704+0000","last_unstale":"2024-09-15T12:24:27.336365+0000","last_undegraded":"2024-09-15T12:24:27.336365+0000","last_fullsized":"2024-09-15T12:24:27.336365+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:24:17.665761+0000","last_change":"2024-09-15T12:24:17.665761+0000","last_active":"2024-09-15T12:24:17.665761+0000","last_peered":"2024-09-15T12:24:17.665761+0000","last_clean":"2024-09-15T12:24:17.665761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:24:17.665761+0000","last_undegraded":"2024-09-15T12:24:17.665761+0000","last_fullsized":"2024-09-15T12:24:17.665761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:24:17.665761+0000","last_change":"2024-09-15T12:24:17.665761+0000","last_active":"2024-09-15T12:24:17.665761+0000","last_peered":"2024-09-15T12:24:17.665761+0000","last_clean":"2024-09-15T12:24:17.665761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:24:17.665761+0000","last_undegraded":"2024-09-15T12:24:17.665761+0000","last_fullsized":"2024-09-15T12:24:17.665761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"36'60","reported_seq":101,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.299036+0000","last_change":"2024-09-15T12:24:23.228012+0000","last_active":"2024-09-15T12:24:27.299036+0000","last_peered":"2024-09-15T12:24:27.299036+0000","last_clean":"2024-09-15T12:24:27.299036+0000","last_became_active":"2024-09-15T12:24:22.193250+0000","last_became_peered":"2024-09-15T12:24:22.193250+0000","last_unstale":"2024-09-15T12:24:27.299036+0000","last_undegraded":"2024-09-15T12:24:27.299036+0000","last_fullsized":"2024-09-15T12:24:27.299036+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":589824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"36'65","reported_seq":113,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.325913+0000","last_change":"2024-09-15T12:24:23.426163+0000","last_active":"2024-09-15T12:24:27.325913+0000","last_peered":"2024-09-15T12:24:27.325913+0000","last_clean":"2024-09-15T12:24:27.325913+0000","last_became_active":"2024-09-15T12:24:22.149793+0000","last_became_peered":"2024-09-15T12:24:22.149793+0000","last_unstale":"2024-09-15T12:24:27.325913+0000","last_undegraded":"2024-09-15T12:24:27.325913+0000","last_fullsized":"2024-09-15T12:24:27.325913+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"36'61","reported_seq":109,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.237444+0000","last_change":"2024-09-15T12:24:23.838853+0000","last_active":"2024-09-15T12:24:27.237444+0000","last_peered":"2024-09-15T12:24:27.237444+0000","last_clean":"2024-09-15T12:24:27.237444+0000","last_became_active":"2024-09-15T12:24:22.187405+0000","last_became_peered":"2024-09-15T12:24:22.187405+0000","last_unstale":"2024-09-15T12:24:27.237444+0000","last_undegraded":"2024-09-15T12:24:27.237444+0000","last_fullsized":"2024-09-15T12:24:27.237444+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3997696,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"36'66","reported_seq":114,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.183457+0000","last_change":"2024-09-15T12:24:26.065855+0000","last_active":"2024-09-15T12:24:27.183457+0000","last_peered":"2024-09-15T12:24:27.183457+0000","last_clean":"2024-09-15T12:24:27.183457+0000","last_became_active":"2024-09-15T12:24:22.238211+0000","last_became_peered":"2024-09-15T12:24:22.238211+0000","last_unstale":"2024-09-15T12:24:27.183457+0000","last_undegraded":"2024-09-15T12:24:27.183457+0000","last_fullsized":"2024-09-15T12:24:27.183457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":91,"reported_epoch":30,"state":"stale+active+undersized+degraded","last_fresh":"2024-09-15T12:23:57.696111+0000","last_change":"2024-09-15T12:23:31.712076+0000","last_active":"2024-09-15T12:23:57.696111+0000","last_peered":"2024-09-15T12:23:57.696111+0000","last_clean":"2024-09-15T12:22:03.910325+0000","last_became_active":"2024-09-15T12:23:31.712076+0000","last_became_peered":"2024-09-15T12:23:31.712076+0000","last_unstale":"2024-09-15T12:24:40.099355+0000","last_undegraded":"2024-09-15T12:23:31.680652+0000","last_fullsized":"2024-09-15T12:23:31.679356+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:21.743011+0000","last_change":"2024-09-15T12:24:16.728919+0000","last_active":"2024-09-15T12:24:21.743011+0000","last_peered":"2024-09-15T12:24:21.743011+0000","last_clean":"2024-09-15T12:24:21.743011+0000","last_became_active":"2024-09-15T12:23:57.793875+0000","last_became_peered":"2024-09-15T12:23:57.793875+0000","last_unstale":"2024-09-15T12:24:21.743011+0000","last_undegraded":"2024-09-15T12:24:21.743011+0000","last_fullsized":"2024-09-15T12:24:21.743011+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:24:17.665761+0000","last_change":"2024-09-15T12:24:17.665761+0000","last_active":"2024-09-15T12:24:17.665761+0000","last_peered":"2024-09-15T12:24:17.665761+0000","last_clean":"2024-09-15T12:24:17.665761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:24:17.665761+0000","last_undegraded":"2024-09-15T12:24:17.665761+0000","last_fullsized":"2024-09-15T12:24:17.665761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":67,"reported_epoch":30,"state":"stale+active+clean","last_fresh":"2024-09-15T12:23:57.826547+0000","last_change":"2024-09-15T12:23:57.827905+0000","last_active":"2024-09-15T12:23:57.826547+0000","last_peered":"2024-09-15T12:23:57.826547+0000","last_clean":"2024-09-15T12:23:31.687384+0000","last_became_active":"2024-09-15T12:23:57.819218+0000","last_became_peered":"2024-09-15T12:23:57.819218+0000","last_unstale":"2024-09-15T12:24:40.099357+0000","last_undegraded":"2024-09-15T12:23:57.826547+0000","last_fullsized":"2024-09-15T12:23:57.826547+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"36'65","reported_seq":111,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.383286+0000","last_change":"2024-09-15T12:24:24.490466+0000","last_active":"2024-09-15T12:24:27.383286+0000","last_peered":"2024-09-15T12:24:27.383286+0000","last_clean":"2024-09-15T12:24:27.383286+0000","last_became_active":"2024-09-15T12:24:22.037357+0000","last_became_peered":"2024-09-15T12:24:22.037357+0000","last_unstale":"2024-09-15T12:24:27.383286+0000","last_undegraded":"2024-09-15T12:24:27.383286+0000","last_fullsized":"2024-09-15T12:24:27.383286+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4259840,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:22.337323+0000","last_change":"2024-09-15T12:24:22.338669+0000","last_active":"2024-09-15T12:24:22.337323+0000","last_peered":"2024-09-15T12:24:22.337323+0000","last_clean":"2024-09-15T12:22:44.815383+0000","last_became_active":"2024-09-15T12:24:22.331340+0000","last_became_peered":"2024-09-15T12:24:22.331340+0000","last_unstale":"2024-09-15T12:24:22.337323+0000","last_undegraded":"2024-09-15T12:24:22.337323+0000","last_fullsized":"2024-09-15T12:24:22.337323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"36'64","reported_seq":111,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.294961+0000","last_change":"2024-09-15T12:24:25.571171+0000","last_active":"2024-09-15T12:24:27.294961+0000","last_peered":"2024-09-15T12:24:27.294961+0000","last_clean":"2024-09-15T12:24:27.294961+0000","last_became_active":"2024-09-15T12:24:22.000784+0000","last_became_peered":"2024-09-15T12:24:22.000784+0000","last_unstale":"2024-09-15T12:24:27.294961+0000","last_undegraded":"2024-09-15T12:24:27.294961+0000","last_fullsized":"2024-09-15T12:24:27.294961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":69,"reported_epoch":30,"state":"stale+active+clean","last_fresh":"2024-09-15T12:23:57.811131+0000","last_change":"2024-09-15T12:23:57.813454+0000","last_active":"2024-09-15T12:23:57.811131+0000","last_peered":"2024-09-15T12:23:57.811131+0000","last_clean":"2024-09-15T12:23:31.682386+0000","last_became_active":"2024-09-15T12:23:57.802720+0000","last_became_peered":"2024-09-15T12:23:57.802720+0000","last_unstale":"2024-09-15T12:24:40.099360+0000","last_undegraded":"2024-09-15T12:23:57.811131+0000","last_fullsized":"2024-09-15T12:23:57.811131+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"36'64","reported_seq":111,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.354502+0000","last_change":"2024-09-15T12:24:25.070438+0000","last_active":"2024-09-15T12:24:27.354502+0000","last_peered":"2024-09-15T12:24:27.354502+0000","last_clean":"2024-09-15T12:24:27.354502+0000","last_became_active":"2024-09-15T12:24:22.021316+0000","last_became_peered":"2024-09-15T12:24:22.021316+0000","last_unstale":"2024-09-15T12:24:27.354502+0000","last_undegraded":"2024-09-15T12:24:27.354502+0000","last_fullsized":"2024-09-15T12:24:27.354502+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":76,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:22.192347+0000","last_change":"2024-09-15T12:24:22.194277+0000","last_active":"2024-09-15T12:24:22.192347+0000","last_peered":"2024-09-15T12:24:22.192347+0000","last_clean":"2024-09-15T12:22:44.802391+0000","last_became_active":"2024-09-15T12:24:22.173329+0000","last_became_peered":"2024-09-15T12:24:22.173329+0000","last_unstale":"2024-09-15T12:24:22.192347+0000","last_undegraded":"2024-09-15T12:24:22.192347+0000","last_fullsized":"2024-09-15T12:24:22.192347+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:24:17.665761+0000","last_change":"2024-09-15T12:24:17.665761+0000","last_active":"2024-09-15T12:24:17.665761+0000","last_peered":"2024-09-15T12:24:17.665761+0000","last_clean":"2024-09-15T12:24:17.665761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:24:17.665761+0000","last_undegraded":"2024-09-15T12:24:17.665761+0000","last_fullsized":"2024-09-15T12:24:17.665761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":30,"state":"active+undersized","last_fresh":"2024-09-15T12:23:57.692132+0000","last_change":"2024-09-15T12:23:31.729846+0000","last_active":"2024-09-15T12:23:57.692132+0000","last_peered":"2024-09-15T12:23:57.692132+0000","last_clean":"2024-09-15T12:22:05.121660+0000","last_became_active":"2024-09-15T12:23:31.729846+0000","last_became_peered":"2024-09-15T12:23:31.729846+0000","last_unstale":"2024-09-15T12:23:57.692132+0000","last_undegraded":"2024-09-15T12:23:57.692132+0000","last_fullsized":"2024-09-15T12:23:31.682106+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:24:17.665761+0000","last_change":"2024-09-15T12:24:17.665761+0000","last_active":"2024-09-15T12:24:17.665761+0000","last_peered":"2024-09-15T12:24:17.665761+0000","last_clean":"2024-09-15T12:24:17.665761+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:24:17.665761+0000","last_undegraded":"2024-09-15T12:24:17.665761+0000","last_fullsized":"2024-09-15T12:24:17.665761+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:22.145635+0000","last_change":"2024-09-15T12:24:22.146730+0000","last_active":"2024-09-15T12:24:22.145635+0000","last_peered":"2024-09-15T12:24:22.145635+0000","last_clean":"2024-09-15T12:22:44.771160+0000","last_became_active":"2024-09-15T12:24:22.127049+0000","last_became_peered":"2024-09-15T12:24:22.127049+0000","last_unstale":"2024-09-15T12:24:22.145635+0000","last_undegraded":"2024-09-15T12:24:22.145635+0000","last_fullsized":"2024-09-15T12:24:22.145635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"36'64","reported_seq":108,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:27.248124+0000","last_change":"2024-09-15T12:24:23.900822+0000","last_active":"2024-09-15T12:24:27.248124+0000","last_peered":"2024-09-15T12:24:27.248124+0000","last_clean":"2024-09-15T12:24:27.248124+0000","last_became_active":"2024-09-15T12:24:22.049810+0000","last_became_peered":"2024-09-15T12:24:22.049810+0000","last_unstale":"2024-09-15T12:24:27.248124+0000","last_undegraded":"2024-09-15T12:24:27.248124+0000","last_fullsized":"2024-09-15T12:24:27.248124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T12:24:21.732644+0000","last_change":"2024-09-15T12:24:16.732586+0000","last_active":"2024-09-15T12:24:21.732644+0000","last_peered":"2024-09-15T12:24:21.732644+0000","last_clean":"2024-09-15T12:24:21.732644+0000","last_became_active":"2024-09-15T12:23:57.798170+0000","last_became_peered":"2024-09-15T12:23:57.798170+0000","last_unstale":"2024-09-15T12:24:21.732644+0000","last_undegraded":"2024-09-15T12:24:21.732644+0000","last_fullsized":"2024-09-15T12:24:21.732644+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":47513600,"num_objects":725,"num_object_clones":0,"num_object_copies":2175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":725,"num_write_kb":46400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":8323072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6348800,"data_stored":101580800,"data_compressed":4822013,"data_compressed_allocated":6348800,"data_compressed_original":101580800,"omap_allocated":0,"internal_metadata":0},"log_size":725,"ondisk_log_size":725,"up":33,"acting":33,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":23,"acting":23,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27032,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":26816,"kb_avail":94344808,"statfs":{"total":96636764160,"available":96609083392,"internally_reserved":0,"allocated":188416,"data_stored":624465,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":27459584},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42412,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94329428,"statfs":{"total":96636764160,"available":96593334272,"internally_reserved":0,"allocated":3174400,"data_stored":45591239,"data_compressed":2161073,"data_compressed_allocated":2846720,"data_compressed_original":45465600,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":29,"seq":124554051614,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43340,"kb_used_data":3324,"kb_used_omap":0,"kb_used_meta":40000,"kb_avail":94328500,"statfs":{"total":96636764160,"available":96592384000,"internally_reserved":0,"allocated":3403776,"data_stored":49653927,"data_compressed":2351825,"data_compressed_allocated":3096576,"data_compressed_original":49545216,"omap_allocated":0,"internal_metadata":40960000},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705783,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50588,"kb_used_data":11276,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":11546624,"data_stored":180312002,"data_compressed":8559689,"data_compressed_allocated":11268096,"data_compressed_original":180207616,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3543040,"data_stored":56688640,"data_compressed":2690996,"data_compressed_allocated":3543040,"data_compressed_original":56688640,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2805760,"data_stored":44892160,"data_compressed":2131017,"data_compressed_allocated":2805760,"data_compressed_original":44892160,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:24:48.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 29 16 2388 2372 5.11152 12.125 0.104324 0.195359 2024-09-15T12:24:49.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:49.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:49.756+0000 7f4b48191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:49.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:49.765+0000 7f4b48191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:49.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 30 16 2589 2573 5.35985 12.5625 0.0475786 0.186058 2024-09-15T12:24:50.772 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 31 16 2808 2792 5.62844 13.6875 0.0518176 0.177444 2024-09-15T12:24:51.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 32 16 3020 3004 5.86657 13.25 0.0942738 0.170254 2024-09-15T12:24:52.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:24:52.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:52.251+0000 7f6df0f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:52.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:52.260+0000 7f6df0f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:52.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 33 16 3236 3220 6.09785 13.5 0.0628012 0.163735 2024-09-15T12:24:53.623 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 1, 3, 0] 2024-09-15T12:24:53.624 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:53.624 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [2, 1, 3, 0] 2024-09-15T12:24:53.624 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T12:24:53.624 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:24:53.702 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T12:24:53.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 34 16 3449 3433 6.31001 13.3125 0.0243899 0.157941 2024-09-15T12:24:54.593 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:24:54.699 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:54.697+0000 7ff5b8a93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:54.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:24:54.702+0000 7ff5b8a93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:24:54.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 35 16 3449 3433 6.12972 0 - 0.157941 2024-09-15T12:24:55.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 36 16 3449 3433 5.95944 0 - 0.157941 2024-09-15T12:24:56.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 37 16 3449 3433 5.79836 0 - 0.157941 2024-09-15T12:24:57.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 38 16 3449 3433 5.64576 0 - 0.157941 2024-09-15T12:24:58.703 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [1, 3, 0] 2024-09-15T12:24:58.703 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:24:58.703 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [1, 3, 0] 2024-09-15T12:24:58.704 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:24:58.704 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:24:58.773 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 39 16 3449 3433 5.501 0 - 0.157941 2024-09-15T12:24:58.787 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:24:59.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:24:59.773299+0000 min lat: 0.0148315 max lat: 15.3117 avg lat: 0.157941 2024-09-15T12:24:59.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:24:59.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 40 16 3449 3433 5.36347 0 - 0.157941 2024-09-15T12:25:00.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 41 16 3449 3433 5.23265 0 - 0.157941 2024-09-15T12:25:01.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 42 16 3449 3433 5.10806 0 - 0.157941 2024-09-15T12:25:02.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 43 16 3449 3433 4.98926 0 - 0.157941 2024-09-15T12:25:03.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 44 16 3449 3433 4.87586 0 - 0.157941 2024-09-15T12:25:03.788 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2, 1] live_osds: [3, 0] 2024-09-15T12:25:03.789 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:25:03.789 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:25:03.789 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:25:03.789 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:25:03.792 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:25:03.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:25:03.905 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:25:03.908 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:03.909 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:25:04.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 45 16 3449 3433 4.76751 0 - 0.157941 2024-09-15T12:25:05.774 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 46 16 3449 3433 4.66387 0 - 0.157941 2024-09-15T12:25:06.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 47 16 3449 3433 4.56464 0 - 0.157941 2024-09-15T12:25:07.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 48 16 3449 3433 4.46954 0 - 0.157941 2024-09-15T12:25:08.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 49 16 3449 3433 4.37833 0 - 0.157941 2024-09-15T12:25:08.910 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:25:09.006 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:25:09.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:09.010 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:25:09.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 50 16 3449 3433 4.29076 0 - 0.157941 2024-09-15T12:25:10.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 51 16 3449 3433 4.20663 0 - 0.157941 2024-09-15T12:25:11.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 52 16 3449 3433 4.12574 0 - 0.157941 2024-09-15T12:25:12.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 53 16 3449 3433 4.04789 0 - 0.157941 2024-09-15T12:25:13.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 54 16 3449 3433 3.97293 0 - 0.157941 2024-09-15T12:25:14.011 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:25:14.120 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:25:14.120 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:25:14.120 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.120 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 53, 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.818250+0000" 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.818275+0000" 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.818744+0000" 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.819102+0000" 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.123 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.820361+0000" 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.124 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821052+0000" 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821399+0000" 2024-09-15T12:25:14.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821580+0000", 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "4.a" 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.127 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.128 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.130 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.131 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 58, 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.133 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.828355+0000" 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.828366+0000" 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.134 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.828588+0000" 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.829144+0000" 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.830355+0000" 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.136 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.137 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.830773+0000" 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.830977+0000" 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.831092+0000", 2024-09-15T12:25:14.139 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "4.d" 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.140 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.144 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 55, 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821655+0000" 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821665+0000" 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.821881+0000" 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.147 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.822060+0000" 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.825695+0000" 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.148 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.826117+0000" 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.826305+0000" 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.826406+0000", 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "4.2" 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.152 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.153 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.156 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.157 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 61, 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.833997+0000" 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.834011+0000" 2024-09-15T12:25:14.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.834205+0000" 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.159 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.834407+0000" 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.835611+0000" 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.161 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.836005+0000" 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.836209+0000", 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-15T12:25:14.163 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1, 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 55 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.165 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.166 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.167 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.168 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.170 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 64, 2024-09-15T12:25:14.171 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.171 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.171 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.837544+0000" 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.837556+0000" 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.172 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.837763+0000" 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.173 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.838541+0000" 2024-09-15T12:25:14.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.839442+0000" 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.839929+0000", 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "ClientRequest::PGPipeline::await_map", 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1, 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 61 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.178 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.180 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.182 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.183 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.184 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 67, 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.841802+0000" 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.841812+0000" 2024-09-15T12:25:14.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.842362+0000" 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.186 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.842613+0000" 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.187 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:25:13.843410+0000", 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1, 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 64 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.188 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:14.191 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.192 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:14.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:25:14.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:25:14.197 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:25:14.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 55 16 3449 3433 3.9007 0 - 0.157941 2024-09-15T12:25:15.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 56 16 3449 3433 3.83105 0 - 0.157941 2024-09-15T12:25:16.775 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 57 16 3449 3433 3.76383 0 - 0.157941 2024-09-15T12:25:17.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 58 16 3449 3433 3.69894 0 - 0.157941 2024-09-15T12:25:18.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 59 16 3449 3433 3.63624 0 - 0.157941 2024-09-15T12:25:19.198 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [3, 0, 1] 2024-09-15T12:25:19.199 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:25:19.199 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-15T12:25:19.199 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T12:25:19.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-15T12:25:19.202 INFO:tasks.ceph.osd.2:Started 2024-09-15T12:25:19.203 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:25:19.314 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:25:19.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:19.318 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:25:19.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:25:19.775744+0000 min lat: 0.0148315 max lat: 15.3117 avg lat: 0.157941 2024-09-15T12:25:19.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:25:19.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 60 16 3449 3433 3.57563 0 - 0.157941 2024-09-15T12:25:20.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 61 16 3449 3433 3.51702 0 - 0.157941 2024-09-15T12:25:21.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 62 16 3449 3433 3.46029 0 - 0.157941 2024-09-15T12:25:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:25:22.243 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:25:22.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:25:22.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 63 16 3449 3433 3.40536 0 - 0.157941 2024-09-15T12:25:23.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 64 16 3538 3522 3.43906 0.185417 0.0558468 0.290612 2024-09-15T12:25:24.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:25:24.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:25:24.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:24.387+0000 7f269db91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:24.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:24.396+0000 7f269db91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:24.438 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:25:24.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:24.442 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:25:24.606 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:25:24.647 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:25:24.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:25:24.776 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 65 16 3693 3677 3.53517 9.6875 0.0577597 0.282618 2024-09-15T12:25:25.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 66 16 3878 3862 3.65678 11.5625 0.0880029 0.273217 2024-09-15T12:25:26.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:25:26.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:26.750+0000 7f708bd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:26.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:26.756+0000 7f708bd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:26.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 67 16 3954 3938 3.67309 4.75 0.0443559 0.269752 2024-09-15T12:25:27.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 68 16 3993 3977 3.65492 2.4375 0.110764 0.273302 2024-09-15T12:25:28.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 69 16 4164 4148 3.75682 10.6875 0.0748404 0.265999 2024-09-15T12:25:28.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:25:29.024 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:29.017+0000 7fe3ff351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:29.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:29.029+0000 7fe3ff351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:29.443 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:25:29.650 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:25:29.650 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [ 2024-09-15T12:25:29.650 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 516, 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.521848+0000" 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.521867+0000" 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.651 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.522062+0000" 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.652 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.522617+0000" 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.522806+0000" 2024-09-15T12:25:29.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.523261+0000" 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.655 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.524000+0000" 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.658 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.524714+0000" 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.659 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.534571+0000" 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 519, 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.544621+0000" 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.544639+0000" 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.544853+0000" 2024-09-15T12:25:29.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.546047+0000" 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.546156+0000" 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.665 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.666 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.546681+0000" 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.667 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.547140+0000" 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.669 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.547698+0000" 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.671 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.552026+0000" 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.672 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.673 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 522, 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.553731+0000" 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.553752+0000" 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.554236+0000" 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.557367+0000" 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.676 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.559336+0000" 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.562751+0000" 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.563049+0000" 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.680 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.567096+0000" 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.682 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.577719+0000" 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.684 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 524, 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.685 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.573234+0000" 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.573252+0000" 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.575080+0000" 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.575873+0000" 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.577963+0000" 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.688 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.579557+0000" 2024-09-15T12:25:29.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.581123+0000" 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.691 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.692 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.583758+0000" 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.693 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.593799+0000" 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.694 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.696 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery", 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 441, 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.f", 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_detail": {} 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 525, 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.697 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 427, 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.897168+0000" 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.897183+0000" 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.698 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.899283+0000" 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.901063+0000" 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.901967+0000" 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.701 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.995229+0000" 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.905447+0000" 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.702 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.905971+0000" 2024-09-15T12:25:29.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.705 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.908700+0000" 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.913112+0000" 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.992772+0000" 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.709 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 432, 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.933503+0000" 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.933518+0000" 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.937658+0000" 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.940066+0000" 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.940795+0000" 2024-09-15T12:25:29.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.002806+0000" 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.944213+0000" 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.944542+0000" 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.715 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.945090+0000" 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.952302+0000" 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.719 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.001853+0000" 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 429, 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.912357+0000" 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.912371+0000" 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.913275+0000" 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.914200+0000" 2024-09-15T12:25:29.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.918199+0000" 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.030723+0000" 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.919244+0000" 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.919889+0000" 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.728 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.920711+0000" 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.931582+0000" 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.026106+0000" 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.732 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 447, 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.004558+0000" 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.004569+0000" 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.005237+0000" 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.006749+0000" 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.735 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.008078+0000" 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.076273+0000" 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.008805+0000" 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.010695+0000" 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.011233+0000" 2024-09-15T12:25:29.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.015261+0000" 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.075140+0000" 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 449, 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.013725+0000" 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.013758+0000" 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.014335+0000" 2024-09-15T12:25:29.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.014467+0000" 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.014599+0000" 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.081674+0000" 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.015776+0000" 2024-09-15T12:25:29.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.016187+0000" 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.750 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.019080+0000" 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.032511+0000" 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.077028+0000" 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 457, 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084198+0000" 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.756 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084210+0000" 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084388+0000" 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084515+0000" 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084730+0000" 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.170250+0000" 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.085132+0000" 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.761 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.085358+0000" 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.085836+0000" 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.090216+0000" 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.167981+0000" 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 458, 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.086131+0000" 2024-09-15T12:25:29.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.086141+0000" 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.086952+0000" 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.087067+0000" 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.087232+0000" 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.191169+0000" 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.772 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.087776+0000" 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.088017+0000" 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.775 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.088488+0000" 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.097099+0000" 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.180255+0000" 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 459, 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.098376+0000" 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.098405+0000" 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.098646+0000" 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.101686+0000" 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.105703+0000" 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.196718+0000" 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.106735+0000" 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.785 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.107977+0000" 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.788 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.109123+0000" 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.115175+0000" 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.194422+0000" 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 462, 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.116377+0000" 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.116393+0000" 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.117318+0000" 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.120221+0000" 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.122940+0000" 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.203561+0000" 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.125062+0000" 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.126364+0000" 2024-09-15T12:25:29.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.799 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.127140+0000" 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.134225+0000" 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.201679+0000" 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.804 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 468, 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.169387+0000" 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.805 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.169398+0000" 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.169751+0000" 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.170374+0000" 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.807 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.170480+0000" 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.227401+0000" 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.170858+0000" 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.171718+0000" 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.811 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.812 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.174351+0000" 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.813 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.178991+0000" 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.814 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.223485+0000" 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.816 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 476, 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.227565+0000" 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.227577+0000" 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.817 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.229445+0000" 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.818 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.229687+0000" 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.819 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.229898+0000" 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.820 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.293136+0000" 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.230513+0000" 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.231721+0000" 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.232962+0000" 2024-09-15T12:25:29.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.236777+0000" 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.827 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.291063+0000" 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.828 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 477, 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.237370+0000" 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.237385+0000" 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.237887+0000" 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.241823+0000" 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.243038+0000" 2024-09-15T12:25:29.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.832 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.310799+0000" 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.245629+0000" 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.833 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.246235+0000" 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.248467+0000" 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.253851+0000" 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.307246+0000" 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 481, 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.251575+0000" 2024-09-15T12:25:29.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.251591+0000" 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.252301+0000" 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.253214+0000" 2024-09-15T12:25:29.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.254095+0000" 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.330027+0000" 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.255127+0000" 2024-09-15T12:25:29.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.845 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.255519+0000" 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.257513+0000" 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.848 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.272630+0000" 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.849 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.325029+0000" 2024-09-15T12:25:29.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 486, 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.851 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.308187+0000" 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.308204+0000" 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.852 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.308605+0000" 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.310502+0000" 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.311233+0000" 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.854 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.407873+0000" 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.311806+0000" 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.856 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.312149+0000" 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.857 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.312673+0000" 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.859 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.860 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.315995+0000" 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.861 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.402032+0000" 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.862 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 489, 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.329292+0000" 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.329304+0000" 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.863 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.330513+0000" 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.335856+0000" 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.865 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.337755+0000" 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.866 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.444619+0000" 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.338252+0000" 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.338522+0000" 2024-09-15T12:25:29.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.869 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.870 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.340439+0000" 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.872 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.346501+0000" 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.439301+0000" 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.874 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 487, 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.314697+0000" 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.314711+0000" 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.315013+0000" 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.315445+0000" 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.316204+0000" 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.878 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.449615+0000" 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.316779+0000" 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.879 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.317895+0000" 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.322116+0000" 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.882 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.883 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.327533+0000" 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.884 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.446929+0000" 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 490, 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.886 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.341592+0000" 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.341606+0000" 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.887 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.342522+0000" 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.342994+0000" 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.892 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.892 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.892 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.343390+0000" 2024-09-15T12:25:29.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.892 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.893 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.457480+0000" 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.347321+0000" 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.894 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.348777+0000" 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.895 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.349931+0000" 2024-09-15T12:25:29.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.364287+0000" 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.454429+0000" 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.900 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 494, 2024-09-15T12:25:29.901 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.366141+0000" 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.366158+0000" 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.367098+0000" 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.369493+0000" 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.904 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.370090+0000" 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.465954+0000" 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.373659+0000" 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.906 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.374672+0000" 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.377187+0000" 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.390219+0000" 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.911 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.464869+0000" 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 508, 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.913 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.486427+0000" 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.486446+0000" 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.914 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.488734+0000" 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.490482+0000" 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.491083+0000" 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.917 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.572795+0000" 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.491689+0000" 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.918 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.492052+0000" 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.919 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.920 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.921 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.493114+0000" 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.498490+0000" 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.923 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.570818+0000" 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.924 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 526, 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.925 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.595974+0000" 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.595993+0000" 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.597634+0000" 2024-09-15T12:25:29.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.600865+0000" 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.606095+0000" 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.928 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.929 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.606672+0000" 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.930 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.607016+0000" 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.931 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.932 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.607674+0000" 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.933 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.609968+0000" 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.936 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 274, 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.937 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.866161+0000" 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.866176+0000" 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.867016+0000" 2024-09-15T12:25:29.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.939 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.868146+0000" 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.870076+0000" 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.940 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.953948+0000" 2024-09-15T12:25:29.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.870534+0000" 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.870778+0000" 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.943 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.944 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.945 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.871213+0000" 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.946 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.873018+0000" 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:27.953053+0000" 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 305, 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.949 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.074944+0000" 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.074961+0000" 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.075371+0000" 2024-09-15T12:25:29.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.075475+0000" 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.078371+0000" 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.952 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.170741+0000" 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.953 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.078776+0000" 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.078989+0000" 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.955 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.079375+0000" 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.957 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.080823+0000" 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.170172+0000" 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.960 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 309, 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.092392+0000" 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.961 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.092405+0000" 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.094276+0000" 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.962 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.095175+0000" 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.963 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.096773+0000" 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.964 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.171823+0000" 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.097323+0000" 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.097528+0000" 2024-09-15T12:25:29.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.968 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.097909+0000" 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.969 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.099228+0000" 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.971 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.171264+0000" 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.972 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 311, 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.109641+0000" 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.109658+0000" 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.973 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.110080+0000" 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.974 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.112168+0000" 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.975 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.115430+0000" 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.176992+0000" 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.115922+0000" 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.977 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.116282+0000" 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.978 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.980 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.116951+0000" 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.981 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.118859+0000" 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.176257+0000" 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.983 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 315, 2024-09-15T12:25:29.984 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.136429+0000" 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.136448+0000" 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.985 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.138386+0000" 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.138729+0000" 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.142101+0000" 2024-09-15T12:25:29.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.988 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.180447+0000" 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.142526+0000" 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.142728+0000" 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.143173+0000" 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.992 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.144699+0000" 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.994 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.179861+0000" 2024-09-15T12:25:29.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 343, 2024-09-15T12:25:29.996 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.324415+0000" 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.324432+0000" 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.325321+0000" 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.325511+0000" 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.329772+0000" 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:29.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.413659+0000" 2024-09-15T12:25:30.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.330222+0000" 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.330423+0000" 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.330808+0000" 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.004 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.332587+0000" 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.412722+0000" 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.007 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 352, 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.388986+0000" 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.008 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.389004+0000" 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.389172+0000" 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.009 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.390956+0000" 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.392630+0000" 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.011 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.505262+0000" 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.393310+0000" 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.393532+0000" 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.013 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.015 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.395861+0000" 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.397799+0000" 2024-09-15T12:25:30.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.504294+0000" 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.019 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 393, 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.653494+0000" 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.653514+0000" 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.020 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.654149+0000" 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.021 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.654297+0000" 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.654726+0000" 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.022 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.763477+0000" 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.023 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.655137+0000" 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.655336+0000" 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.025 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.026 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.655759+0000" 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.027 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.028 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.660770+0000" 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.029 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.762834+0000" 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.030 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 434, 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.965753+0000" 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.031 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.965771+0000" 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.965997+0000" 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.966113+0000" 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.033 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.966709+0000" 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.034 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.076130+0000" 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.967135+0000" 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.036 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.967333+0000" 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.037 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.967722+0000" 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.039 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.969244+0000" 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.075345+0000" 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 436, 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.970576+0000" 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.043 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.970589+0000" 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.971205+0000" 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.044 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.971374+0000" 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.972939+0000" 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.046 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.077099+0000" 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.973762+0000" 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.047 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.974730+0000" 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.976950+0000" 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.982168+0000" 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.074758+0000" 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.053 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.075587+0000" 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.054 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 444, 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.996518+0000" 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.996539+0000" 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.998234+0000" 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.998476+0000" 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.999105+0000" 2024-09-15T12:25:30.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.058 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.084711+0000" 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.999517+0000" 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.059 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:28.999718+0000" 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.060 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.061 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.000128+0000" 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.001646+0000" 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.064 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.083910+0000" 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.065 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 464, 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.129117+0000" 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.066 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.129133+0000" 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.129747+0000" 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.131143+0000" 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.068 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.131655+0000" 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.231613+0000" 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.132087+0000" 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.070 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.132297+0000" 2024-09-15T12:25:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.072 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.132692+0000" 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.074 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.134192+0000" 2024-09-15T12:25:30.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.230701+0000" 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.077 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 471, 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.188146+0000" 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.188164+0000" 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.078 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.189817+0000" 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.079 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.191568+0000" 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.193816+0000" 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.278416+0000" 2024-09-15T12:25:30.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.194449+0000" 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.194885+0000" 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.083 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.196958+0000" 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.086 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.199586+0000" 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.277822+0000" 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 472, 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.195988+0000" 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.196007+0000" 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.196995+0000" 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.090 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.197796+0000" 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.091 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.203212+0000" 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.314870+0000" 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.203768+0000" 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.203993+0000" 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.204948+0000" 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.207452+0000" 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.098 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.314294+0000" 2024-09-15T12:25:30.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.100 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 473, 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.200895+0000" 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.200925+0000" 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.101 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.201728+0000" 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.202582+0000" 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.203273+0000" 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.103 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.318479+0000" 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.204918+0000" 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.205760+0000" 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.206162+0000" 2024-09-15T12:25:30.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.109 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.209103+0000" 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.110 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.317936+0000" 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 484, 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.283223+0000" 2024-09-15T12:25:30.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.283235+0000" 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.283757+0000" 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.284249+0000" 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.290014+0000" 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.373977+0000" 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.290683+0000" 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.291027+0000" 2024-09-15T12:25:30.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.119 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.291681+0000" 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.294498+0000" 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.122 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.373416+0000" 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 492, 2024-09-15T12:25:30.123 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.353762+0000" 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.353791+0000" 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.355463+0000" 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.125 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.359069+0000" 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.363184+0000" 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.126 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.467013+0000" 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.363734+0000" 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.128 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.363975+0000" 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.129 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.130 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.364488+0000" 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.368673+0000" 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.463363+0000" 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.134 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 502, 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.459364+0000" 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.459383+0000" 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.459531+0000" 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.136 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.459659+0000" 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.137 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.461079+0000" 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.546708+0000" 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.461518+0000" 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.461714+0000" 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.462137+0000" 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.143 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.470302+0000" 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.144 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.545723+0000" 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.145 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 511, 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.499637+0000" 2024-09-15T12:25:30.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.499649+0000" 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.506095+0000" 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.506352+0000" 2024-09-15T12:25:30.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.515134+0000" 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.149 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.598589+0000" 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.515959+0000" 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.150 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.516408+0000" 2024-09-15T12:25:30.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.152 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.153 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.517174+0000" 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.154 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.522911+0000" 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.155 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:25:29.598010+0000" 2024-09-15T12:25:30.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:25:30.157 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:25:30.158 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 70 16 4331 4315 3.85225 10.4375 0.048569 0.259392 2024-09-15T12:25:30.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 71 16 4499 4483 3.94586 10.5 0.0984163 0.253174 2024-09-15T12:25:31.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:25:31.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:31.650+0000 7ff377d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:31.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:31.658+0000 7ff377d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:31.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 72 16 4670 4654 4.03948 10.6875 0.12156 0.247407 2024-09-15T12:25:32.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 73 16 4823 4807 4.11512 9.5625 0.124032 0.242813 2024-09-15T12:25:33.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 74 16 4969 4953 4.18281 9.125 0.149888 0.238825 2024-09-15T12:25:34.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:25:34.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:34.259+0000 7f8b4cb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:34.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:34.268+0000 7f8b4cb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:34.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 75 16 5135 5119 4.26536 10.375 0.0803981 0.234219 2024-09-15T12:25:35.159 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:25:35.160 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:25:35.160 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:25:35.160 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:25:35.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.307+0000 7f42010c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.323+0000 7f42010c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.402 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:35.402 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:35.403 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:35.406 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:35.407 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:35.407 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:25:35.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.564+0000 7f67c0d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.580+0000 7f67c0d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:35.733 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:35.734 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:35.734 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:35.734 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:35.734 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:35.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:35.738 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:35.738 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:25:35.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 76 16 5314 5298 4.35643 11.1875 0.0453706 0.229406 2024-09-15T12:25:35.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.883+0000 7f7f95911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:35.894+0000 7f7f95911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:35.985 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:35.985 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:35.986 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:25:36.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:36.186+0000 7f9e4cd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:36.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:36.210+0000 7f9e4cd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:36.245 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:36.246 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:36.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:36.250 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:36.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:25:36.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:36.771+0000 7fa84c0c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:36.777 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 77 16 5498 5482 4.44918 11.5 0.0536742 0.224632 2024-09-15T12:25:36.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:36.778+0000 7fa84c0c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:37.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 78 16 5669 5653 4.52915 10.6875 0.0479399 0.220658 2024-09-15T12:25:38.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 79 16 5851 5835 4.61579 11.375 0.111958 0.216533 2024-09-15T12:25:39.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:25:39.334 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:39.330+0000 7f2ad9aa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:39.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:39.340+0000 7f2ad9aa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:39.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:25:39.777618+0000 min lat: 0.0148315 max lat: 29.674 avg lat: 0.213467 2024-09-15T12:25:39.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:25:39.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 80 16 6008 5992 4.68074 9.8125 0.168962 0.213467 2024-09-15T12:25:40.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 81 16 6191 6175 4.76414 11.4375 0.104799 0.209754 2024-09-15T12:25:41.251 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:25:41.251 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:25:41.251 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [3, 0, 1, 2] 2024-09-15T12:25:41.252 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T12:25:41.252 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:25:41.340 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T12:25:41.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:25:41.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:41.754+0000 7f55f8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:41.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:41.759+0000 7f55f8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:41.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 82 16 6288 6272 4.77997 6.0625 0.0424677 0.208017 2024-09-15T12:25:42.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 83 16 6288 6272 4.72238 0 - 0.208017 2024-09-15T12:25:43.778 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 84 16 6288 6272 4.66616 0 - 0.208017 2024-09-15T12:25:44.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 85 16 6288 6272 4.61126 0 - 0.208017 2024-09-15T12:25:45.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 86 16 6288 6272 4.55763 0 - 0.208017 2024-09-15T12:25:46.341 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [3, 0, 1] 2024-09-15T12:25:46.341 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:25:46.341 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:25:46.341 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:25:46.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.457+0000 7f765065c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.467+0000 7f765065c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:46.483 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:46.484 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:46.484 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:46.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:46.486 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:46.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:25:46.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.589+0000 7fc93aaa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.599+0000 7fc93aaa3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.614 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:25:46.614 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:25:46.614 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:25:46.614 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:25:46.615 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:25:46.618 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:25:46.618 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:25:46.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:25:46.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.723+0000 7f7aa1751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:25:46.732+0000 7f7aa1751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:25:46.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 87 16 6288 6272 4.50525 0 - 0.208017 2024-09-15T12:25:47.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 88 16 6288 6272 4.45405 0 - 0.208017 2024-09-15T12:25:48.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 89 16 6288 6272 4.404 0 - 0.208017 2024-09-15T12:25:49.779 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 90 16 6288 6272 4.35507 0 - 0.208017 2024-09-15T12:25:50.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 91 16 6288 6272 4.30721 0 - 0.208017 2024-09-15T12:25:51.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 92 16 6288 6272 4.26039 0 - 0.208017 2024-09-15T12:25:52.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 93 16 6288 6272 4.21458 0 - 0.208017 2024-09-15T12:25:53.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 94 16 6288 6272 4.16974 0 - 0.208017 2024-09-15T12:25:54.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 95 16 6288 6272 4.12585 0 - 0.208017 2024-09-15T12:25:55.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 96 16 6288 6272 4.08287 0 - 0.208017 2024-09-15T12:25:56.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 97 16 6288 6272 4.04078 0 - 0.208017 2024-09-15T12:25:57.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 98 16 6288 6272 3.99955 0 - 0.208017 2024-09-15T12:25:58.780 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 99 16 6288 6272 3.95915 0 - 0.208017 2024-09-15T12:25:59.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:25:59.780263+0000 min lat: 0.0148315 max lat: 29.674 avg lat: 0.208017 2024-09-15T12:25:59.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:25:59.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 100 16 6288 6272 3.91955 0 - 0.208017 2024-09-15T12:26:00.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 101 16 6288 6272 3.88074 0 - 0.208017 2024-09-15T12:26:01.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 102 16 6288 6272 3.8427 0 - 0.208017 2024-09-15T12:26:02.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 103 16 6288 6272 3.80539 0 - 0.208017 2024-09-15T12:26:03.781 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 104 16 6288 6272 3.7688 0 - 0.208017 2024-09-15T12:26:04.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 105 16 6288 6272 3.7329 0 - 0.208017 2024-09-15T12:26:05.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 106 16 6288 6272 3.69769 0 - 0.208017 2024-09-15T12:26:06.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 107 16 6288 6272 3.66313 0 - 0.208017 2024-09-15T12:26:07.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 108 16 6288 6272 3.62921 0 - 0.208017 2024-09-15T12:26:08.670 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:08.671 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:08.674 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:08.674 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:26:08.675 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:26:08.778 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:08.779 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:08.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 109 16 6288 6272 3.59591 0 - 0.208017 2024-09-15T12:26:08.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:08.831+0000 7fd15bb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:08.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:08.840+0000 7fd15bb91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:26:08.858 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:26:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:26:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:26:08.859 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:26:08.861 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:26:08.862 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:26:09.782 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 110 14 6288 6274 3.56436 0.00446429 28.5682 0.217053 2024-09-15T12:26:10.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total time run: 110.126 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total writes made: 6288 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Write size: 65536 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object size: 65536 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Bandwidth (MB/sec): 3.56865 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Bandwidth: 5.26308 2024-09-15T12:26:10.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max bandwidth (MB/sec): 14 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min bandwidth (MB/sec): 0 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average IOPS: 57 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev IOPS: 84.2207 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max IOPS: 224 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min IOPS: 0 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average Latency(s): 0.280062 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Latency(s): 2.19757 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max latency(s): 29.674 2024-09-15T12:26:10.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min latency(s): 0.0148315 2024-09-15T12:26:10.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T12:26:10.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:10.931+0000 7f7eabefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:10.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:10.939+0000 7f7eabefd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:11.145 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:11.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:11.182 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:13.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:26:13.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:13.313+0000 7fdcd5a88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:13.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:13.319+0000 7fdcd5a88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:13.516 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:13.545 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:13.546 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:13.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [3, 0, 1] 2024-09-15T12:26:13.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:26:13.863 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-15T12:26:13.863 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T12:26:13.863 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-15T12:26:13.866 INFO:tasks.ceph.osd.2:Started 2024-09-15T12:26:13.866 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:13.990 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:26:13.993 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:26:13.993 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:26:15.547 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:15.717 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:15.713+0000 7f84cae4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:15.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:15.719+0000 7f84cae4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:15.959 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:16.024 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:16.024 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:18.025 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:26:18.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:18.153+0000 7f93f0f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:18.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:18.162+0000 7f93f0f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:18.366 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:18.436 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:18.437 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:18.994 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:19.139 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:26:19.143 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:26:19.143 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:26:20.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:20.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:20.583+0000 7f3b43f08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:20.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:20.592+0000 7f3b43f08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:20.818 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:20.907 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:20.907 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:22.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:26:23.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:23.068+0000 7f01f5d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:23.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:23.076+0000 7f01f5d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:23.273 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:23.323 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:23.324 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:24.144 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:24.257 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:26:24.257 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [] 2024-09-15T12:26:24.257 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:26:25.324 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:25.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:25.427+0000 7fba1e6a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:25.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:25.432+0000 7fba1e6a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:27.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:26:27.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:27.664+0000 7ff144108640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:27.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:27.670+0000 7ff144108640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:29.267 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:26:29.268 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:26:29.268 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [3, 0, 1, 2] 2024-09-15T12:26:29.268 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T12:26:29.268 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:26:30.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:30.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:30.132+0000 7f81b5863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:30.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:30.138+0000 7f81b5863640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:33.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T12:26:35.370 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T12:26:38.808 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-15T12:26:40.371 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [3, 0, 1] 2024-09-15T12:26:40.371 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:26:40.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:26:40.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:40.489+0000 7fdb8a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:40.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:40.497+0000 7fdb8a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:40.685 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:26:40.685 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:26:40.698 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-15T12:26:40.063667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":370934347,"num_objects":5655,"num_object_clones":0,"num_object_copies":16965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3542,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":6963,"num_write_kb":402197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":25034752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6960,"ondisk_log_size":6960,"up":56,"acting":56,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":199440,"kb_used_data":56332,"kb_used_omap":0,"kb_used_meta":143040,"kb_avail":377287920,"statfs":{"total":386547056640,"available":386342830080,"internally_reserved":0,"allocated":57683968,"data_stored":897949614,"data_compressed":42616735,"data_compressed_allocated":56098816,"data_compressed_original":897335296,"omap_allocated":0,"internal_metadata":146472960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003360"},"pg_stats":[{"pgid":"4.f","version":"48'560","reported_seq":656,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:21.127163+0000","last_change":"2024-09-15T12:26:09.764811+0000","last_active":"2024-09-15T12:26:21.127163+0000","last_peered":"2024-09-15T12:26:21.127163+0000","last_clean":"2024-09-15T12:25:41.002286+0000","last_became_active":"2024-09-15T12:26:09.764811+0000","last_became_peered":"2024-09-15T12:26:09.764811+0000","last_unstale":"2024-09-15T12:26:21.127163+0000","last_undegraded":"2024-09-15T12:26:09.706552+0000","last_fullsized":"2024-09-15T12:26:09.705361+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17104896,"num_objects":261,"num_object_clones":0,"num_object_copies":783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":557,"num_write_kb":26176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":261}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"48'422","reported_seq":534,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:12.642570+0000","last_change":"2024-09-15T12:26:09.768141+0000","last_active":"2024-09-15T12:26:12.642570+0000","last_peered":"2024-09-15T12:26:12.642570+0000","last_clean":"2024-09-15T12:25:41.198562+0000","last_became_active":"2024-09-15T12:26:09.768141+0000","last_became_peered":"2024-09-15T12:26:09.768141+0000","last_unstale":"2024-09-15T12:26:12.642570+0000","last_undegraded":"2024-09-15T12:26:09.706201+0000","last_fullsized":"2024-09-15T12:26:09.704913+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":420,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"48'416","reported_seq":561,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:12.508134+0000","last_change":"2024-09-15T12:26:09.828582+0000","last_active":"2024-09-15T12:26:12.508134+0000","last_peered":"2024-09-15T12:26:12.508134+0000","last_clean":"2024-09-15T12:25:41.379986+0000","last_became_active":"2024-09-15T12:26:09.828582+0000","last_became_peered":"2024-09-15T12:26:09.828582+0000","last_unstale":"2024-09-15T12:26:12.508134+0000","last_undegraded":"2024-09-15T12:26:09.720988+0000","last_fullsized":"2024-09-15T12:26:09.719132+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510488,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":415,"num_write_kb":25217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2818048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":375}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"48'520","reported_seq":595,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:21.297459+0000","last_change":"2024-09-15T12:25:14.781493+0000","last_active":"2024-09-15T12:26:21.297459+0000","last_peered":"2024-09-15T12:26:21.297459+0000","last_clean":"2024-09-15T12:26:21.297459+0000","last_became_active":"2024-09-15T12:25:14.775366+0000","last_became_peered":"2024-09-15T12:25:14.775366+0000","last_unstale":"2024-09-15T12:26:21.297459+0000","last_undegraded":"2024-09-15T12:26:21.297459+0000","last_fullsized":"2024-09-15T12:26:21.297459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16121856,"num_objects":246,"num_object_clones":0,"num_object_copies":738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"48'389","reported_seq":533,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:10.866183+0000","last_change":"2024-09-15T12:26:09.824413+0000","last_active":"2024-09-15T12:26:10.866183+0000","last_peered":"2024-09-15T12:26:10.866183+0000","last_clean":"2024-09-15T12:25:41.335394+0000","last_became_active":"2024-09-15T12:26:09.824413+0000","last_became_peered":"2024-09-15T12:26:09.824413+0000","last_unstale":"2024-09-15T12:26:10.866183+0000","last_undegraded":"2024-09-15T12:26:09.716682+0000","last_fullsized":"2024-09-15T12:26:09.715393+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":378,"num_object_clones":0,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":378,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":388,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":378}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"48'419","reported_seq":577,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:12.553877+0000","last_change":"2024-09-15T12:26:09.817252+0000","last_active":"2024-09-15T12:26:12.553877+0000","last_peered":"2024-09-15T12:26:12.553877+0000","last_clean":"2024-09-15T12:25:41.324586+0000","last_became_active":"2024-09-15T12:26:09.817252+0000","last_became_peered":"2024-09-15T12:26:09.817252+0000","last_unstale":"2024-09-15T12:26:12.553877+0000","last_undegraded":"2024-09-15T12:26:09.724987+0000","last_fullsized":"2024-09-15T12:26:09.723050+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":384,"num_object_clones":0,"num_object_copies":1152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":384,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":418,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":384}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"48'426","reported_seq":517,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:12.696004+0000","last_change":"2024-09-15T12:25:14.810817+0000","last_active":"2024-09-15T12:26:12.696004+0000","last_peered":"2024-09-15T12:26:12.696004+0000","last_clean":"2024-09-15T12:26:12.696004+0000","last_became_active":"2024-09-15T12:25:14.802778+0000","last_became_peered":"2024-09-15T12:25:14.802778+0000","last_unstale":"2024-09-15T12:26:12.696004+0000","last_undegraded":"2024-09-15T12:26:12.696004+0000","last_fullsized":"2024-09-15T12:26:12.696004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24707072,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":425,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"48'410","reported_seq":499,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:12.585830+0000","last_change":"2024-09-15T12:25:14.826105+0000","last_active":"2024-09-15T12:26:12.585830+0000","last_peered":"2024-09-15T12:26:12.585830+0000","last_clean":"2024-09-15T12:26:12.585830+0000","last_became_active":"2024-09-15T12:25:14.820925+0000","last_became_peered":"2024-09-15T12:25:14.820925+0000","last_unstale":"2024-09-15T12:26:12.585830+0000","last_undegraded":"2024-09-15T12:26:12.585830+0000","last_fullsized":"2024-09-15T12:26:12.585830+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":409,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"48'427","reported_seq":511,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:12.725610+0000","last_change":"2024-09-15T12:25:14.778592+0000","last_active":"2024-09-15T12:26:12.725610+0000","last_peered":"2024-09-15T12:26:12.725610+0000","last_clean":"2024-09-15T12:26:12.725610+0000","last_became_active":"2024-09-15T12:25:14.768899+0000","last_became_peered":"2024-09-15T12:25:14.768899+0000","last_unstale":"2024-09-15T12:26:12.725610+0000","last_undegraded":"2024-09-15T12:26:12.725610+0000","last_fullsized":"2024-09-15T12:26:12.725610+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25034752,"num_objects":382,"num_object_clones":0,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":426,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":137,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:09.760851+0000","last_change":"2024-09-15T12:26:09.760194+0000","last_active":"2024-09-15T12:26:09.760851+0000","last_peered":"2024-09-15T12:26:09.760851+0000","last_clean":"2024-09-15T12:25:27.271128+0000","last_became_active":"2024-09-15T12:26:09.760194+0000","last_became_peered":"2024-09-15T12:26:09.760194+0000","last_unstale":"2024-09-15T12:26:09.760851+0000","last_undegraded":"2024-09-15T12:26:09.703245+0000","last_fullsized":"2024-09-15T12:26:09.701971+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":83,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.821115+0000","last_change":"2024-09-15T12:26:09.819966+0000","last_active":"2024-09-15T12:26:09.821115+0000","last_peered":"2024-09-15T12:26:09.821115+0000","last_clean":"2024-09-15T12:25:27.274624+0000","last_became_active":"2024-09-15T12:26:09.819966+0000","last_became_peered":"2024-09-15T12:26:09.819966+0000","last_unstale":"2024-09-15T12:26:09.821115+0000","last_undegraded":"2024-09-15T12:26:09.821115+0000","last_fullsized":"2024-09-15T12:26:09.718900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"48'398","reported_seq":540,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:10.965889+0000","last_change":"2024-09-15T12:26:09.812359+0000","last_active":"2024-09-15T12:26:10.965889+0000","last_peered":"2024-09-15T12:26:10.965889+0000","last_clean":"2024-09-15T12:25:41.327491+0000","last_became_active":"2024-09-15T12:26:09.812359+0000","last_became_peered":"2024-09-15T12:26:09.812359+0000","last_unstale":"2024-09-15T12:26:10.965889+0000","last_undegraded":"2024-09-15T12:26:09.721302+0000","last_fullsized":"2024-09-15T12:26:09.719506+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":391,"num_object_clones":0,"num_object_copies":1173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":391,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":397,"num_write_kb":25216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2555904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":391}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":102,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.814847+0000","last_change":"2024-09-15T12:26:09.814133+0000","last_active":"2024-09-15T12:26:09.814847+0000","last_peered":"2024-09-15T12:26:09.814847+0000","last_clean":"2024-09-15T12:25:27.315732+0000","last_became_active":"2024-09-15T12:26:09.814133+0000","last_became_peered":"2024-09-15T12:26:09.814133+0000","last_unstale":"2024-09-15T12:26:09.814847+0000","last_undegraded":"2024-09-15T12:26:09.814847+0000","last_fullsized":"2024-09-15T12:26:09.719773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"48'381","reported_seq":490,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:10.830153+0000","last_change":"2024-09-15T12:26:09.782634+0000","last_active":"2024-09-15T12:26:10.830153+0000","last_peered":"2024-09-15T12:26:10.830153+0000","last_clean":"2024-09-15T12:25:41.021520+0000","last_became_active":"2024-09-15T12:26:09.782634+0000","last_became_peered":"2024-09-15T12:26:09.782634+0000","last_unstale":"2024-09-15T12:26:10.830153+0000","last_undegraded":"2024-09-15T12:26:09.706974+0000","last_fullsized":"2024-09-15T12:26:09.705325+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24576000,"num_objects":375,"num_object_clones":0,"num_object_copies":1125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":375,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":379,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":1048576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":70,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.837218+0000","last_change":"2024-09-15T12:26:09.836113+0000","last_active":"2024-09-15T12:26:09.837218+0000","last_peered":"2024-09-15T12:26:09.837218+0000","last_clean":"2024-09-15T12:25:27.275995+0000","last_became_active":"2024-09-15T12:26:09.836113+0000","last_became_peered":"2024-09-15T12:26:09.836113+0000","last_unstale":"2024-09-15T12:26:09.837218+0000","last_undegraded":"2024-09-15T12:26:09.837218+0000","last_fullsized":"2024-09-15T12:26:09.728487+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"48'530","reported_seq":634,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:21.322418+0000","last_change":"2024-09-15T12:26:09.770641+0000","last_active":"2024-09-15T12:26:21.322418+0000","last_peered":"2024-09-15T12:26:21.322418+0000","last_clean":"2024-09-15T12:25:41.092998+0000","last_became_active":"2024-09-15T12:26:09.770641+0000","last_became_peered":"2024-09-15T12:26:09.770641+0000","last_unstale":"2024-09-15T12:26:21.322418+0000","last_undegraded":"2024-09-15T12:26:09.708919+0000","last_fullsized":"2024-09-15T12:26:09.707811+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17498112,"num_objects":267,"num_object_clones":0,"num_object_copies":801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":267,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":527,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":267}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":104,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:09.804184+0000","last_change":"2024-09-15T12:26:09.803525+0000","last_active":"2024-09-15T12:26:09.804184+0000","last_peered":"2024-09-15T12:26:09.804184+0000","last_clean":"2024-09-15T12:25:27.292174+0000","last_became_active":"2024-09-15T12:26:09.803525+0000","last_became_peered":"2024-09-15T12:26:09.803525+0000","last_unstale":"2024-09-15T12:26:09.804184+0000","last_undegraded":"2024-09-15T12:26:09.711626+0000","last_fullsized":"2024-09-15T12:26:09.710375+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"48'420","reported_seq":576,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:12.533553+0000","last_change":"2024-09-15T12:26:09.811729+0000","last_active":"2024-09-15T12:26:12.533553+0000","last_peered":"2024-09-15T12:26:12.533553+0000","last_clean":"2024-09-15T12:25:41.365043+0000","last_became_active":"2024-09-15T12:26:09.811729+0000","last_became_peered":"2024-09-15T12:26:09.811729+0000","last_unstale":"2024-09-15T12:26:12.533553+0000","last_undegraded":"2024-09-15T12:26:09.727430+0000","last_fullsized":"2024-09-15T12:26:09.725503+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23789568,"num_objects":363,"num_object_clones":0,"num_object_copies":1089,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":3145728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":363}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.776047+0000","last_change":"2024-09-15T12:26:09.775449+0000","last_active":"2024-09-15T12:26:09.776047+0000","last_peered":"2024-09-15T12:26:09.776047+0000","last_clean":"2024-09-15T12:24:22.194277+0000","last_became_active":"2024-09-15T12:26:09.775449+0000","last_became_peered":"2024-09-15T12:26:09.775449+0000","last_unstale":"2024-09-15T12:26:09.776047+0000","last_undegraded":"2024-09-15T12:26:09.776047+0000","last_fullsized":"2024-09-15T12:26:09.716726+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"48'390","reported_seq":511,"reported_epoch":48,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:26:10.979764+0000","last_change":"2024-09-15T12:26:09.818902+0000","last_active":"2024-09-15T12:26:10.979764+0000","last_peered":"2024-09-15T12:26:10.979764+0000","last_clean":"2024-09-15T12:25:41.075003+0000","last_became_active":"2024-09-15T12:26:09.818902+0000","last_became_peered":"2024-09-15T12:26:09.818902+0000","last_unstale":"2024-09-15T12:26:10.979764+0000","last_undegraded":"2024-09-15T12:26:09.717579+0000","last_fullsized":"2024-09-15T12:26:09.716346+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":379,"num_object_clones":0,"num_object_copies":1137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":389,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":72,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.798106+0000","last_change":"2024-09-15T12:26:09.797479+0000","last_active":"2024-09-15T12:26:09.798106+0000","last_peered":"2024-09-15T12:26:09.798106+0000","last_clean":"2024-09-15T12:25:27.275299+0000","last_became_active":"2024-09-15T12:26:09.797479+0000","last_became_peered":"2024-09-15T12:26:09.797479+0000","last_unstale":"2024-09-15T12:26:09.798106+0000","last_undegraded":"2024-09-15T12:26:09.798106+0000","last_fullsized":"2024-09-15T12:26:09.715010+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"48'390","reported_seq":468,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:09.729118+0000","last_change":"2024-09-15T12:25:14.784723+0000","last_active":"2024-09-15T12:26:09.729118+0000","last_peered":"2024-09-15T12:26:09.729118+0000","last_clean":"2024-09-15T12:26:09.729118+0000","last_became_active":"2024-09-15T12:25:14.778535+0000","last_became_peered":"2024-09-15T12:25:14.778535+0000","last_unstale":"2024-09-15T12:26:09.729118+0000","last_undegraded":"2024-09-15T12:26:09.729118+0000","last_fullsized":"2024-09-15T12:26:09.729118+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":387,"num_object_clones":0,"num_object_copies":1161,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":389,"num_write_kb":24832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.801591+0000","last_change":"2024-09-15T12:26:09.800872+0000","last_active":"2024-09-15T12:26:09.801591+0000","last_peered":"2024-09-15T12:26:09.801591+0000","last_clean":"2024-09-15T12:24:22.146730+0000","last_became_active":"2024-09-15T12:26:09.800872+0000","last_became_peered":"2024-09-15T12:26:09.800872+0000","last_unstale":"2024-09-15T12:26:09.801591+0000","last_undegraded":"2024-09-15T12:26:09.801591+0000","last_fullsized":"2024-09-15T12:26:09.723148+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"48'407","reported_seq":487,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-15T12:26:12.702534+0000","last_change":"2024-09-15T12:25:14.797795+0000","last_active":"2024-09-15T12:26:12.702534+0000","last_peered":"2024-09-15T12:26:12.702534+0000","last_clean":"2024-09-15T12:26:12.702534+0000","last_became_active":"2024-09-15T12:25:14.788676+0000","last_became_peered":"2024-09-15T12:25:14.788676+0000","last_unstale":"2024-09-15T12:26:12.702534+0000","last_undegraded":"2024-09-15T12:26:12.702534+0000","last_fullsized":"2024-09-15T12:26:12.702534+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":406,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.795895+0000","last_change":"2024-09-15T12:26:09.795231+0000","last_active":"2024-09-15T12:26:09.795895+0000","last_peered":"2024-09-15T12:26:09.795895+0000","last_clean":"2024-09-15T12:25:27.289822+0000","last_became_active":"2024-09-15T12:26:09.795231+0000","last_became_peered":"2024-09-15T12:26:09.795231+0000","last_unstale":"2024-09-15T12:26:09.795895+0000","last_undegraded":"2024-09-15T12:26:09.795895+0000","last_fullsized":"2024-09-15T12:26:09.708004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":370343960,"num_objects":5652,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6884,"num_write_kb":401089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":25034752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55463936,"data_stored":887291952,"data_compressed":42131409,"data_compressed_allocated":55455744,"data_compressed_original":887291904,"omap_allocated":0,"internal_metadata":0},"log_size":6905,"ondisk_log_size":6905,"up":38,"acting":38,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50384,"kb_used_data":14460,"kb_used_omap":0,"kb_used_meta":35904,"kb_avail":94321456,"statfs":{"total":96636764160,"available":96585170944,"internally_reserved":0,"allocated":14807040,"data_stored":230249027,"data_compressed":10928280,"data_compressed_allocated":14385152,"data_compressed_original":230080512,"omap_allocated":0,"internal_metadata":36765696},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42412,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94329428,"statfs":{"total":96636764160,"available":96593334272,"internally_reserved":0,"allocated":3174400,"data_stored":45591239,"data_compressed":2161073,"data_compressed_allocated":2846720,"data_compressed_original":45465600,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47760,"kb_used_data":19520,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94324080,"statfs":{"total":96636764160,"available":96587857920,"internally_reserved":0,"allocated":19988480,"data_stored":313413710,"data_compressed":14874889,"data_compressed_allocated":19578880,"data_compressed_original":313262080,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757337,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58884,"kb_used_data":19252,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94312956,"statfs":{"total":96636764160,"available":96576466944,"internally_reserved":0,"allocated":19714048,"data_stored":308695638,"data_compressed":14652493,"data_compressed_allocated":19288064,"data_compressed_original":308527104,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19136512,"data_stored":306118680,"data_compressed":14535364,"data_compressed_allocated":19132416,"data_compressed_original":306118656,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19570688,"data_stored":313065496,"data_compressed":14865556,"data_compressed_allocated":19566592,"data_compressed_original":313065472,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2805760,"data_stored":44892160,"data_compressed":2131017,"data_compressed_allocated":2805760,"data_compressed_original":44892160,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13950976,"data_stored":223215616,"data_compressed":10599472,"data_compressed_allocated":13950976,"data_compressed_original":223215616,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:26:40.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 4.f 4.e 4.d 4.b 2.2 4.2 4.3 2024-09-15T12:26:40.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:40.803+0000 7f804dd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:40.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:40.818+0000 7f804dd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:41.004 INFO:teuthology.orchestra.run.smithi103.stderr:instructing pg(s) [2.2,4.3,4.b,4.f] on osd.0 to force-recovery; instructing pg(s) [4.2,4.d,4.e] on osd.1 to force-recovery; 2024-09-15T12:26:41.004 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:26:46.018 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [2] live_osds: [3, 0, 1] 2024-09-15T12:26:46.018 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:26:46.019 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-15T12:26:46.019 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T12:26:46.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-15T12:26:46.022 INFO:tasks.ceph.osd.2:Started 2024-09-15T12:26:46.022 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:46.133 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:26:46.136 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:26:46.137 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:26:51.137 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:51.234 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:26:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:26:51.237 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:26:55.137 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:26:55.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:26:55.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:26:56.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:26:56.410 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:26:56.410 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [ 2024-09-15T12:26:56.410 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 87, 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.7", 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 53, 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 53, 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 53 epoch_requested: 53 MLogRec from 0 log log((51'569,51'572], crt=51'572) pi ([0,0] all_participants={} intervals=[]) +create_info", 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378028+0000" 2024-09-15T12:26:56.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378038+0000" 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378219+0000" 2024-09-15T12:26:56.412 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378479+0000" 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378604+0000" 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.414 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378752+0000" 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.415 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.378824+0000" 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.416 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 84, 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.3", 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 53, 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 53, 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 53 epoch_requested: 53 MQuery 4.3 from 0 query_epoch 53 query: query(log 46'0 ec=32/32 lis/c=50/45 les/c/f=51/46/0 sis=53 pruub=10.836426735s epoch_sent 53)", 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.365792+0000" 2024-09-15T12:26:56.418 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.365802+0000" 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.365993+0000" 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.419 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.366147+0000" 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.367155+0000" 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:26:56.421 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.367312+0000" 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.367368+0000" 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 86, 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:26:56.424 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.4", 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 53, 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 53, 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 53 epoch_requested: 53 MLogRec from 1 log log((51'590,51'592], crt=51'592) pi ([0,0] all_participants={} intervals=[]) +create_info", 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.372124+0000" 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.372133+0000" 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.425 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.372471+0000" 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.375153+0000" 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.427 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.382151+0000" 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.382336+0000" 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.382379+0000" 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:26:56.430 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 85, 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "0", 2024-09-15T12:26:56.431 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "4.b", 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 53, 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 53, 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 53 epoch_requested: 53 MQuery 4.b from 0 query_epoch 53 query: query(log 46'0 ec=32/32 lis/c=50/45 les/c/f=51/46/0 sis=53 pruub=10.808115959s epoch_sent 53)", 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.366872+0000" 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.366877+0000" 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.432 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.367979+0000" 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:26:56.433 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.369171+0000" 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.434 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.382233+0000" 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.435 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.391586+0000" 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:26:56.391667+0000" 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.437 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:26:56.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.438 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:26:56.438 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:26:56.438 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:26:57.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:26:57.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:57.389+0000 7f478b591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:57.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:57.395+0000 7f478b591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:59.717 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:26:59.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:59.882+0000 7f549f191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:26:59.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:26:59.890+0000 7f549f191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:01.439 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:27:01.439 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:01.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:27:01.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:01.593+0000 7fe8fdf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:01.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:01.608+0000 7fe8fdf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:01.888 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:01.888 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:27:01.908 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-15T12:27:00.068740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204669515,"num_objects":3118,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":9500,"num_write_kb":402197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":691,"num_bytes_recovered":25427992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9498,"ondisk_log_size":9498,"up":73,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":181480,"kb_used_data":34216,"kb_used_omap":0,"kb_used_meta":147200,"kb_avail":377305880,"statfs":{"total":386547056640,"available":386361221120,"internally_reserved":0,"allocated":35037184,"data_stored":534389984,"data_compressed":25352681,"data_compressed_allocated":33374208,"data_compressed_original":533741568,"omap_allocated":0,"internal_metadata":150732800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.935306"},"pg_stats":[{"pgid":"4.f","version":"51'622","reported_seq":829,"reported_epoch":51,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T12:26:29.250425+0000","last_change":"2024-09-15T12:26:27.258502+0000","last_active":"2024-09-15T12:26:29.250425+0000","last_peered":"2024-09-15T12:26:29.250425+0000","last_clean":"2024-09-15T12:25:41.002286+0000","last_became_active":"2024-09-15T12:26:26.971586+0000","last_became_peered":"2024-09-15T12:26:26.971586+0000","last_unstale":"2024-09-15T12:26:29.250425+0000","last_undegraded":"2024-09-15T12:26:29.250425+0000","last_fullsized":"2024-09-15T12:26:26.688424+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13107200,"num_objects":200,"num_object_clones":0,"num_object_copies":600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":618,"num_write_kb":26176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":200}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"51'605","reported_seq":739,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.313830+0000","last_change":"2024-09-15T12:26:27.200713+0000","last_active":"2024-09-15T12:26:29.313830+0000","last_peered":"2024-09-15T12:26:29.313830+0000","last_clean":"2024-09-15T12:25:41.198562+0000","last_became_active":"2024-09-15T12:26:26.981303+0000","last_became_peered":"2024-09-15T12:26:26.981303+0000","last_unstale":"2024-09-15T12:26:29.313830+0000","last_undegraded":"2024-09-15T12:26:26.709042+0000","last_fullsized":"2024-09-15T12:26:26.689595+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11993088,"num_objects":183,"num_object_clones":0,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":1245184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":183}],"blocked_by":[],"up_primary":2,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"51'591","reported_seq":845,"reported_epoch":51,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T12:26:29.275443+0000","last_change":"2024-09-15T12:26:27.374131+0000","last_active":"2024-09-15T12:26:29.275443+0000","last_peered":"2024-09-15T12:26:29.275443+0000","last_clean":"2024-09-15T12:25:41.379986+0000","last_became_active":"2024-09-15T12:26:27.063053+0000","last_became_peered":"2024-09-15T12:26:27.063053+0000","last_unstale":"2024-09-15T12:26:29.275443+0000","last_undegraded":"2024-09-15T12:26:29.275443+0000","last_fullsized":"2024-09-15T12:26:26.754551+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041688,"num_objects":200,"num_object_clones":0,"num_object_copies":600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":590,"num_write_kb":25217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":90,"num_bytes_recovered":2883608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":200}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"51'578","reported_seq":662,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.202877+0000","last_change":"2024-09-15T12:25:14.781493+0000","last_active":"2024-09-15T12:26:29.202877+0000","last_peered":"2024-09-15T12:26:29.202877+0000","last_clean":"2024-09-15T12:26:29.202877+0000","last_became_active":"2024-09-15T12:25:14.775366+0000","last_became_peered":"2024-09-15T12:25:14.775366+0000","last_unstale":"2024-09-15T12:26:29.202877+0000","last_undegraded":"2024-09-15T12:26:29.202877+0000","last_fullsized":"2024-09-15T12:26:29.202877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12320768,"num_objects":188,"num_object_clones":0,"num_object_copies":564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":578,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"51'564","reported_seq":718,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.216389+0000","last_change":"2024-09-15T12:26:27.070072+0000","last_active":"2024-09-15T12:26:29.216389+0000","last_peered":"2024-09-15T12:26:29.216389+0000","last_clean":"2024-09-15T12:25:41.335394+0000","last_became_active":"2024-09-15T12:26:27.061776+0000","last_became_peered":"2024-09-15T12:26:27.061776+0000","last_unstale":"2024-09-15T12:26:29.216389+0000","last_undegraded":"2024-09-15T12:26:26.759811+0000","last_fullsized":"2024-09-15T12:26:26.742865+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13303808,"num_objects":203,"num_object_clones":0,"num_object_copies":609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":160,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":563,"num_write_kb":24512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":200}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"51'596","reported_seq":764,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.254777+0000","last_change":"2024-09-15T12:26:27.073891+0000","last_active":"2024-09-15T12:26:29.254777+0000","last_peered":"2024-09-15T12:26:29.254777+0000","last_clean":"2024-09-15T12:25:41.324586+0000","last_became_active":"2024-09-15T12:26:27.055441+0000","last_became_peered":"2024-09-15T12:26:27.055441+0000","last_unstale":"2024-09-15T12:26:29.254777+0000","last_undegraded":"2024-09-15T12:26:26.750670+0000","last_fullsized":"2024-09-15T12:26:26.720946+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13565952,"num_objects":207,"num_object_clones":0,"num_object_copies":621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3211264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":205}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"51'617","reported_seq":717,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.142655+0000","last_change":"2024-09-15T12:25:14.810817+0000","last_active":"2024-09-15T12:26:29.142655+0000","last_peered":"2024-09-15T12:26:29.142655+0000","last_clean":"2024-09-15T12:26:29.142655+0000","last_became_active":"2024-09-15T12:25:14.802778+0000","last_became_peered":"2024-09-15T12:25:14.802778+0000","last_unstale":"2024-09-15T12:26:29.142655+0000","last_undegraded":"2024-09-15T12:26:29.142655+0000","last_fullsized":"2024-09-15T12:26:29.142655+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12189696,"num_objects":186,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":616,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"51'582","reported_seq":680,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.018112+0000","last_change":"2024-09-15T12:25:14.826105+0000","last_active":"2024-09-15T12:26:29.018112+0000","last_peered":"2024-09-15T12:26:29.018112+0000","last_clean":"2024-09-15T12:26:29.018112+0000","last_became_active":"2024-09-15T12:25:14.820925+0000","last_became_peered":"2024-09-15T12:25:14.820925+0000","last_unstale":"2024-09-15T12:26:29.018112+0000","last_undegraded":"2024-09-15T12:26:29.018112+0000","last_fullsized":"2024-09-15T12:26:29.018112+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12517376,"num_objects":191,"num_object_clones":0,"num_object_copies":573,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":581,"num_write_kb":24704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"51'598","reported_seq":691,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.293794+0000","last_change":"2024-09-15T12:25:14.778592+0000","last_active":"2024-09-15T12:26:29.293794+0000","last_peered":"2024-09-15T12:26:29.293794+0000","last_clean":"2024-09-15T12:26:29.293794+0000","last_became_active":"2024-09-15T12:25:14.768899+0000","last_became_peered":"2024-09-15T12:25:14.768899+0000","last_unstale":"2024-09-15T12:26:29.293794+0000","last_undegraded":"2024-09-15T12:26:29.293794+0000","last_fullsized":"2024-09-15T12:26:29.293794+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13828096,"num_objects":211,"num_object_clones":0,"num_object_copies":633,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":597,"num_write_kb":25856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":149,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.686699+0000","last_change":"2024-09-15T12:26:25.829925+0000","last_active":"2024-09-15T12:26:26.686699+0000","last_peered":"2024-09-15T12:26:26.686699+0000","last_clean":"2024-09-15T12:26:26.686699+0000","last_became_active":"2024-09-15T12:26:25.819082+0000","last_became_peered":"2024-09-15T12:26:25.819082+0000","last_unstale":"2024-09-15T12:26:26.686699+0000","last_undegraded":"2024-09-15T12:26:26.686699+0000","last_fullsized":"2024-09-15T12:26:26.686699+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.737472+0000","last_change":"2024-09-15T12:26:26.740861+0000","last_active":"2024-09-15T12:26:26.737472+0000","last_peered":"2024-09-15T12:26:26.737472+0000","last_clean":"2024-09-15T12:26:26.737472+0000","last_became_active":"2024-09-15T12:26:25.812821+0000","last_became_peered":"2024-09-15T12:26:25.812821+0000","last_unstale":"2024-09-15T12:26:26.737472+0000","last_undegraded":"2024-09-15T12:26:26.737472+0000","last_fullsized":"2024-09-15T12:26:26.737472+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"51'594","reported_seq":746,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.279380+0000","last_change":"2024-09-15T12:26:27.037408+0000","last_active":"2024-09-15T12:26:29.279380+0000","last_peered":"2024-09-15T12:26:29.279380+0000","last_clean":"2024-09-15T12:25:41.327491+0000","last_became_active":"2024-09-15T12:26:27.028498+0000","last_became_peered":"2024-09-15T12:26:27.028498+0000","last_unstale":"2024-09-15T12:26:29.279380+0000","last_undegraded":"2024-09-15T12:26:26.781715+0000","last_fullsized":"2024-09-15T12:26:26.764203+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":195,"num_object_clones":0,"num_object_copies":585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":25216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2555904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":193}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.743276+0000","last_change":"2024-09-15T12:26:26.767345+0000","last_active":"2024-09-15T12:26:26.743276+0000","last_peered":"2024-09-15T12:26:26.743276+0000","last_clean":"2024-09-15T12:26:26.743276+0000","last_became_active":"2024-09-15T12:26:25.869049+0000","last_became_peered":"2024-09-15T12:26:25.869049+0000","last_unstale":"2024-09-15T12:26:26.743276+0000","last_undegraded":"2024-09-15T12:26:26.743276+0000","last_fullsized":"2024-09-15T12:26:26.743276+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"51'572","reported_seq":695,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.272659+0000","last_change":"2024-09-15T12:26:27.101976+0000","last_active":"2024-09-15T12:26:29.272659+0000","last_peered":"2024-09-15T12:26:29.272659+0000","last_clean":"2024-09-15T12:25:41.021520+0000","last_became_active":"2024-09-15T12:26:26.963095+0000","last_became_peered":"2024-09-15T12:26:26.963095+0000","last_unstale":"2024-09-15T12:26:29.272659+0000","last_undegraded":"2024-09-15T12:26:26.738502+0000","last_fullsized":"2024-09-15T12:26:26.711965+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12058624,"num_objects":184,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":570,"num_write_kb":24128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[{"shards":"0,3","objects":184}],"blocked_by":[],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.718800+0000","last_change":"2024-09-15T12:26:26.847126+0000","last_active":"2024-09-15T12:26:26.718800+0000","last_peered":"2024-09-15T12:26:26.718800+0000","last_clean":"2024-09-15T12:26:26.718800+0000","last_became_active":"2024-09-15T12:26:25.816136+0000","last_became_peered":"2024-09-15T12:26:25.816136+0000","last_unstale":"2024-09-15T12:26:26.718800+0000","last_undegraded":"2024-09-15T12:26:26.718800+0000","last_fullsized":"2024-09-15T12:26:26.718800+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"51'592","reported_seq":891,"reported_epoch":51,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T12:26:29.291433+0000","last_change":"2024-09-15T12:26:27.114476+0000","last_active":"2024-09-15T12:26:29.291433+0000","last_peered":"2024-09-15T12:26:29.291433+0000","last_clean":"2024-09-15T12:25:41.092998+0000","last_became_active":"2024-09-15T12:26:26.991301+0000","last_became_peered":"2024-09-15T12:26:26.991301+0000","last_unstale":"2024-09-15T12:26:29.291433+0000","last_undegraded":"2024-09-15T12:26:29.291433+0000","last_fullsized":"2024-09-15T12:26:26.689613+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13434880,"num_objects":205,"num_object_clones":0,"num_object_copies":615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":589,"num_write_kb":25408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":129,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":205}],"blocked_by":[],"up_primary":2,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":117,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.741366+0000","last_change":"2024-09-15T12:26:26.759006+0000","last_active":"2024-09-15T12:26:26.741366+0000","last_peered":"2024-09-15T12:26:26.741366+0000","last_clean":"2024-09-15T12:26:26.741366+0000","last_became_active":"2024-09-15T12:26:25.847302+0000","last_became_peered":"2024-09-15T12:26:25.847302+0000","last_unstale":"2024-09-15T12:26:26.741366+0000","last_undegraded":"2024-09-15T12:26:26.741366+0000","last_fullsized":"2024-09-15T12:26:26.741366+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"51'585","reported_seq":751,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.325038+0000","last_change":"2024-09-15T12:26:27.048265+0000","last_active":"2024-09-15T12:26:29.325038+0000","last_peered":"2024-09-15T12:26:29.325038+0000","last_clean":"2024-09-15T12:25:41.365043+0000","last_became_active":"2024-09-15T12:26:27.044078+0000","last_became_peered":"2024-09-15T12:26:27.044078+0000","last_unstale":"2024-09-15T12:26:29.325038+0000","last_undegraded":"2024-09-15T12:26:26.734726+0000","last_fullsized":"2024-09-15T12:26:26.716936+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":584,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":3145728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":197}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.776047+0000","last_change":"2024-09-15T12:26:09.775449+0000","last_active":"2024-09-15T12:26:09.776047+0000","last_peered":"2024-09-15T12:26:09.776047+0000","last_clean":"2024-09-15T12:24:22.194277+0000","last_became_active":"2024-09-15T12:26:09.775449+0000","last_became_peered":"2024-09-15T12:26:09.775449+0000","last_unstale":"2024-09-15T12:26:09.776047+0000","last_undegraded":"2024-09-15T12:26:09.776047+0000","last_fullsized":"2024-09-15T12:26:09.716726+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"51'589","reported_seq":723,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:26:29.229001+0000","last_change":"2024-09-15T12:26:27.097234+0000","last_active":"2024-09-15T12:26:29.229001+0000","last_peered":"2024-09-15T12:26:29.229001+0000","last_clean":"2024-09-15T12:25:41.075003+0000","last_became_active":"2024-09-15T12:26:27.087453+0000","last_became_peered":"2024-09-15T12:26:27.087453+0000","last_unstale":"2024-09-15T12:26:29.229001+0000","last_undegraded":"2024-09-15T12:26:26.762774+0000","last_fullsized":"2024-09-15T12:26:26.744948+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11796480,"num_objects":180,"num_object_clones":0,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":588,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":2162688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.760765+0000","last_change":"2024-09-15T12:26:26.764169+0000","last_active":"2024-09-15T12:26:26.760765+0000","last_peered":"2024-09-15T12:26:26.760765+0000","last_clean":"2024-09-15T12:26:26.760765+0000","last_became_active":"2024-09-15T12:26:25.843795+0000","last_became_peered":"2024-09-15T12:26:25.843795+0000","last_unstale":"2024-09-15T12:26:26.760765+0000","last_undegraded":"2024-09-15T12:26:26.760765+0000","last_fullsized":"2024-09-15T12:26:26.760765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"51'571","reported_seq":658,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.125849+0000","last_change":"2024-09-15T12:25:14.784723+0000","last_active":"2024-09-15T12:26:29.125849+0000","last_peered":"2024-09-15T12:26:29.125849+0000","last_clean":"2024-09-15T12:26:29.125849+0000","last_became_active":"2024-09-15T12:25:14.778535+0000","last_became_peered":"2024-09-15T12:25:14.778535+0000","last_unstale":"2024-09-15T12:26:29.125849+0000","last_undegraded":"2024-09-15T12:26:29.125849+0000","last_fullsized":"2024-09-15T12:26:29.125849+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13500416,"num_objects":206,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":570,"num_write_kb":24832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":48,"state":"active+undersized","last_fresh":"2024-09-15T12:26:09.801591+0000","last_change":"2024-09-15T12:26:09.800872+0000","last_active":"2024-09-15T12:26:09.801591+0000","last_peered":"2024-09-15T12:26:09.801591+0000","last_clean":"2024-09-15T12:24:22.146730+0000","last_became_active":"2024-09-15T12:26:09.800872+0000","last_became_peered":"2024-09-15T12:26:09.800872+0000","last_unstale":"2024-09-15T12:26:09.801591+0000","last_undegraded":"2024-09-15T12:26:09.801591+0000","last_fullsized":"2024-09-15T12:26:09.723148+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"51'587","reported_seq":676,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:29.045207+0000","last_change":"2024-09-15T12:25:14.797795+0000","last_active":"2024-09-15T12:26:29.045207+0000","last_peered":"2024-09-15T12:26:29.045207+0000","last_clean":"2024-09-15T12:26:29.045207+0000","last_became_active":"2024-09-15T12:25:14.788676+0000","last_became_peered":"2024-09-15T12:25:14.788676+0000","last_unstale":"2024-09-15T12:26:29.045207+0000","last_undegraded":"2024-09-15T12:26:29.045207+0000","last_fullsized":"2024-09-15T12:26:29.045207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:24:17.665761+0000","last_clean_scrub_stamp":"2024-09-15T12:24:17.665761+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11665408,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":586,"num_write_kb":24448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T12:26:26.713383+0000","last_change":"2024-09-15T12:26:26.720988+0000","last_active":"2024-09-15T12:26:26.713383+0000","last_peered":"2024-09-15T12:26:26.713383+0000","last_clean":"2024-09-15T12:26:26.713383+0000","last_became_active":"2024-09-15T12:26:25.787893+0000","last_became_peered":"2024-09-15T12:26:25.787893+0000","last_unstale":"2024-09-15T12:26:26.713383+0000","last_undegraded":"2024-09-15T12:26:26.713383+0000","last_fullsized":"2024-09-15T12:26:26.713383+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":204079128,"num_objects":3115,"num_object_clones":0,"num_object_copies":9345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9421,"num_write_kb":401089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":691,"num_bytes_recovered":25427992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":33259520,"data_stored":532021296,"data_compressed":25262513,"data_compressed_allocated":33251328,"data_compressed_original":532021248,"omap_allocated":0,"internal_metadata":0},"log_size":9443,"ondisk_log_size":9443,"up":48,"acting":38,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":22,"acting":22,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50752,"kb_used_data":10672,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94321088,"statfs":{"total":96636764160,"available":96584794112,"internally_reserved":0,"allocated":10928128,"data_stored":167804577,"data_compressed":7962544,"data_compressed_allocated":10481664,"data_compressed_original":167624704,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42412,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":39296,"kb_avail":94329428,"statfs":{"total":96636764160,"available":96593334272,"internally_reserved":0,"allocated":3174400,"data_stored":45591239,"data_compressed":2161073,"data_compressed_allocated":2846720,"data_compressed_original":45465600,"omap_allocated":0,"internal_metadata":40239104},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47184,"kb_used_data":10624,"kb_used_omap":0,"kb_used_meta":36544,"kb_avail":94324656,"statfs":{"total":96636764160,"available":96588447744,"internally_reserved":0,"allocated":10878976,"data_stored":167279788,"data_compressed":7935600,"data_compressed_allocated":10444800,"data_compressed_original":167116800,"omap_allocated":0,"internal_metadata":37421056},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757367,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41132,"kb_used_data":9820,"kb_used_omap":0,"kb_used_meta":31296,"kb_avail":94330708,"statfs":{"total":96636764160,"available":96594644992,"internally_reserved":0,"allocated":10055680,"data_stored":153714380,"data_compressed":7293464,"data_compressed_allocated":9601024,"data_compressed_original":153534464,"omap_allocated":0,"internal_metadata":32047104},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9564160,"data_stored":152961048,"data_compressed":7263408,"data_compressed_allocated":9560064,"data_compressed_original":152961024,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10448896,"data_stored":167116824,"data_compressed":7935600,"data_compressed_allocated":10444800,"data_compressed_original":167116800,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2805760,"data_stored":44892160,"data_compressed":2131017,"data_compressed_allocated":2805760,"data_compressed_original":44892160,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10440704,"data_stored":167051264,"data_compressed":7932488,"data_compressed_allocated":10440704,"data_compressed_original":167051264,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:27:01.909 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 4.f 4.e 4.b 4.a 4.6 4.7 4.2 4.3 2024-09-15T12:27:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:02.047+0000 7fc146711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:02.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:02.073+0000 7fc146711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:02.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:02.433 INFO:teuthology.orchestra.run.smithi103.stderr:instructing pg(s) [4.3,4.6,4.b,4.f] on osd.0 to force-recovery; instructing pg(s) [4.2,4.a] on osd.1 to force-recovery; instructing pg(s) [4.7,4.e] on osd.2 to force-recovery; 2024-09-15T12:27:02.434 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:27:02.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:02.447+0000 7f16d2c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:02.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:02.455+0000 7f16d2c4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:04.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:05.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:05.042+0000 7f4677711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:05.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:05.050+0000 7f4677711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.456 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:27:07.456 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:07.456 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:27:07.457 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:27:07.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:07.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.578+0000 7fd959391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.590+0000 7fd959391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:07.624 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:07.625 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:07.628 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:07.629 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:07.629 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:27:07.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.622+0000 7f132a391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.632+0000 7f132a391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.798+0000 7fc0a66ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:07.816+0000 7fc0a66ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:07.869 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:07.870 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:07.873 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:07.874 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:07.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:27:08.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:08.050+0000 7f7573191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:08.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:08.062+0000 7f7573191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:08.218 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:08.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:08.221 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:08.221 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:27:08.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:08.355+0000 7f19d2d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:08.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:08.366+0000 7f19d2d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:08.393 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:08.394 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:08.395 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:08.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:08.397 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:09.993 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:10.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:10.153+0000 7fb733b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:10.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:10.165+0000 7fb733b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:12.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:12.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:12.720+0000 7f51f22e3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:12.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:12.728+0000 7f51f22e3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:13.398 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:27:13.398 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:13.398 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:27:13.399 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:27:13.578 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:13.568+0000 7f3d46708640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:13.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:13.582+0000 7f3d46708640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:13.613 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:13.613 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:13.614 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:13.615 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:13.618 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:13.618 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:13.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:27:13.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:13.760+0000 7feef4191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:13.773+0000 7feef4191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:13.936 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:13.937 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:13.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:13.942 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:13.942 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:27:14.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:14.086+0000 7f65f3f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:14.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:14.098+0000 7f65f3f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:14.282 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:14.283 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:14.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:14.288 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:14.288 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:27:14.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:14.439+0000 7fbcc907f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:14.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:14.467+0000 7fbcc907f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:14.509 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:14.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:14.513 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:15.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:15.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:15.332+0000 7faa2fea3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:15.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:15.341+0000 7faa2fea3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:15.560 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Removed 6288 objects 2024-09-15T12:27:15.560 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Clean up completed and total clean up time :64.7371 2024-09-15T12:27:16.640 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-15T12:27:16.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-15T12:27:16.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:16.764+0000 7f8e3f191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:16.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:16.777+0000 7f8e3f191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:17.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:17.682 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_1' removed 2024-09-15T12:27:17.705 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-15T12:27:17.706 INFO:tasks.radosbench:Beginning radosbench... 2024-09-15T12:27:17.707 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-15T12:27:17.707 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_2 16 2024-09-15T12:27:17.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:17.808+0000 7f6c1fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:17.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:17.814+0000 7f6c1fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:17.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:17.886+0000 7fcd2ed91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:17.903 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:17.902+0000 7fcd2ed91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:18.690 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_2' created 2024-09-15T12:27:18.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-15T12:27:18.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:18.839+0000 7f671584f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:18.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:18.853+0000 7f671584f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.514 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:27:19.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:19.514 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:27:19.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:27:19.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.618+0000 7f4447591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.627+0000 7f4447591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:19.643 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:19.644 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:19.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:19.646 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:19.647 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:27:19.693 INFO:teuthology.orchestra.run.smithi103.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-15T12:27:19.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.779+0000 7fd5ceb11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.793+0000 7fd5ceb11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:19.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:19.821 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:19.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:27:19.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.974+0000 7f4cc3111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:19.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:19.983+0000 7f4cc3111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:20.000 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:20.000 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:20.000 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:20.000 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:20.000 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:20.001 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:20.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:20.003 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:20.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:27:20.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:20.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:20.105+0000 7f03f8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:20.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:20.114+0000 7f03f8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:20.132 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:27:20.133 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:27:20.136 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:20.136 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:27:20.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:20.211+0000 7faa26791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:20.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:20.216+0000 7faa26791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:20.717 DEBUG:teuthology.orchestra.run.smithi103:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-15T12:27:20.720 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-15T12:27:20.720 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-15T12:27:20.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:hints = 1 2024-09-15T12:27:20.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-15T12:27:20.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object prefix: benchmark_data_smithi103_35798 2024-09-15T12:27:20.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:27:20.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 0 0 0 0 0 0 - 0 2024-09-15T12:27:21.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1 16 224 208 12.9993 13 0.0903097 0.0734783 2024-09-15T12:27:22.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:22.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:22.459+0000 7fc4734bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:22.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:22.468+0000 7fc4734bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:22.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2 16 455 439 13.7178 14.4375 0.0756336 0.0720743 2024-09-15T12:27:23.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3 16 677 661 13.7698 13.875 0.0957278 0.0715796 2024-09-15T12:27:24.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:24.783 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 4 16 908 892 13.9364 14.4375 0.103312 0.0710286 2024-09-15T12:27:24.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:24.878+0000 7fd4c0351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:24.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:24.885+0000 7fd4c0351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:25.137 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [3, 0, 1, 2] 2024-09-15T12:27:25.138 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:25.138 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [3, 0, 1, 2] 2024-09-15T12:27:25.138 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:27:25.138 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:27:25.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 5 16 1000 984 12.2989 5.75 0.0391937 0.0715028 2024-09-15T12:27:26.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 6 16 1000 984 10.249 0 - 0.0715028 2024-09-15T12:27:27.192 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:27.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:27.307+0000 7f804ef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:27.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:27.313+0000 7f804ef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:27.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 7 16 1000 984 8.7848 0 - 0.0715028 2024-09-15T12:27:28.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 8 16 1000 984 7.68665 0 - 0.0715028 2024-09-15T12:27:29.050 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-15T12:27:29.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 9 16 1000 984 6.83257 0 - 0.0715028 2024-09-15T12:27:30.784 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 10 16 1000 984 6.14929 0 - 0.0715028 2024-09-15T12:27:31.239 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:27:31.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 11 16 1000 984 5.59024 0 - 0.0715028 2024-09-15T12:27:32.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 12 16 1000 984 5.12438 0 - 0.0715028 2024-09-15T12:27:33.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 13 16 1000 984 4.73018 0 - 0.0715028 2024-09-15T12:27:34.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-15T12:27:34.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 14 16 1000 984 4.3923 0 - 0.0715028 2024-09-15T12:27:35.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 15 16 1000 984 4.09948 0 - 0.0715028 2024-09-15T12:27:36.240 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [3, 0, 2] 2024-09-15T12:27:36.240 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:27:36.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:27:36.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:36.358+0000 7fd279a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:36.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:36.366+0000 7fd279a4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:36.551 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:36.551 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:27:36.564 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-15T12:27:36.076633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27394611,"num_objects":412,"num_object_clones":0,"num_object_copies":1236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":488,"num_write_kb":27284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":464,"ondisk_log_size":464,"up":51,"acting":51,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":146916,"kb_used_data":10852,"kb_used_omap":0,"kb_used_meta":136000,"kb_avail":377340444,"statfs":{"total":386547056640,"available":386396614656,"internally_reserved":0,"allocated":11112448,"data_stored":146491327,"data_compressed":6926229,"data_compressed_allocated":9121792,"data_compressed_original":145702912,"omap_allocated":0,"internal_metadata":139264000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":393,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002652"},"pg_stats":[{"pgid":"5.f","version":"59'67","reported_seq":79,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.963908+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:24.963908+0000","last_peered":"2024-09-15T12:27:24.963908+0000","last_clean":"2024-09-15T12:27:24.963908+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:24.963908+0000","last_undegraded":"2024-09-15T12:27:24.963908+0000","last_fullsized":"2024-09-15T12:27:24.963908+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:21.127514+0000","last_change":"2024-09-15T12:27:19.823556+0000","last_active":"2024-09-15T12:27:21.127514+0000","last_peered":"2024-09-15T12:27:21.127514+0000","last_clean":"2024-09-15T12:27:21.127514+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:21.127514+0000","last_undegraded":"2024-09-15T12:27:21.127514+0000","last_fullsized":"2024-09-15T12:27:21.127514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.168379+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:25.168379+0000","last_peered":"2024-09-15T12:27:25.168379+0000","last_clean":"2024-09-15T12:27:25.168379+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:25.168379+0000","last_undegraded":"2024-09-15T12:27:25.168379+0000","last_fullsized":"2024-09-15T12:27:25.168379+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.059287+0000","last_change":"2024-09-15T12:27:19.907587+0000","last_active":"2024-09-15T12:27:25.059287+0000","last_peered":"2024-09-15T12:27:25.059287+0000","last_clean":"2024-09-15T12:27:25.059287+0000","last_became_active":"2024-09-15T12:27:19.901757+0000","last_became_peered":"2024-09-15T12:27:19.901757+0000","last_unstale":"2024-09-15T12:27:25.059287+0000","last_undegraded":"2024-09-15T12:27:25.059287+0000","last_fullsized":"2024-09-15T12:27:25.059287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":174,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.710311+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:20.710311+0000","last_peered":"2024-09-15T12:27:20.710311+0000","last_clean":"2024-09-15T12:27:20.710311+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:20.710311+0000","last_undegraded":"2024-09-15T12:27:20.710311+0000","last_fullsized":"2024-09-15T12:27:20.710311+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:26:58.166967+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":129,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T12:26:58.172670+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:26:58.172670+0000","last_peered":"2024-09-15T12:26:58.172670+0000","last_clean":"2024-09-15T12:26:58.172670+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:26:58.172670+0000","last_undegraded":"2024-09-15T12:26:58.172670+0000","last_fullsized":"2024-09-15T12:26:58.172670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"59'67","reported_seq":79,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.046025+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:25.046025+0000","last_peered":"2024-09-15T12:27:25.046025+0000","last_clean":"2024-09-15T12:27:25.046025+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:25.046025+0000","last_undegraded":"2024-09-15T12:27:25.046025+0000","last_fullsized":"2024-09-15T12:27:25.046025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":67,"num_object_clones":0,"num_object_copies":201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":67,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:26:58.178043+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"59'63","reported_seq":75,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.123024+0000","last_change":"2024-09-15T12:27:19.845490+0000","last_active":"2024-09-15T12:27:25.123024+0000","last_peered":"2024-09-15T12:27:25.123024+0000","last_clean":"2024-09-15T12:27:25.123024+0000","last_became_active":"2024-09-15T12:27:19.836513+0000","last_became_peered":"2024-09-15T12:27:19.836513+0000","last_unstale":"2024-09-15T12:27:25.123024+0000","last_undegraded":"2024-09-15T12:27:25.123024+0000","last_fullsized":"2024-09-15T12:27:25.123024+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":131,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T12:26:58.162138+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:26:58.162138+0000","last_peered":"2024-09-15T12:26:58.162138+0000","last_clean":"2024-09-15T12:26:58.162138+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:26:58.162138+0000","last_undegraded":"2024-09-15T12:26:58.162138+0000","last_fullsized":"2024-09-15T12:26:58.162138+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"59'69","reported_seq":81,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.213241+0000","last_change":"2024-09-15T12:27:19.850978+0000","last_active":"2024-09-15T12:27:25.213241+0000","last_peered":"2024-09-15T12:27:25.213241+0000","last_clean":"2024-09-15T12:27:25.213241+0000","last_became_active":"2024-09-15T12:27:19.839948+0000","last_became_peered":"2024-09-15T12:27:19.839948+0000","last_unstale":"2024-09-15T12:27:25.213241+0000","last_undegraded":"2024-09-15T12:27:25.213241+0000","last_fullsized":"2024-09-15T12:27:25.213241+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":69,"num_object_clones":0,"num_object_copies":207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":69,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.717882+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:20.717882+0000","last_peered":"2024-09-15T12:27:20.717882+0000","last_clean":"2024-09-15T12:27:20.717882+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:20.717882+0000","last_undegraded":"2024-09-15T12:27:20.717882+0000","last_fullsized":"2024-09-15T12:27:20.717882+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.715418+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:20.715418+0000","last_peered":"2024-09-15T12:27:20.715418+0000","last_clean":"2024-09-15T12:27:20.715418+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:20.715418+0000","last_undegraded":"2024-09-15T12:27:20.715418+0000","last_fullsized":"2024-09-15T12:27:20.715418+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:21.260896+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:21.260896+0000","last_peered":"2024-09-15T12:27:21.260896+0000","last_clean":"2024-09-15T12:27:21.260896+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:21.260896+0000","last_undegraded":"2024-09-15T12:27:21.260896+0000","last_fullsized":"2024-09-15T12:27:21.260896+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.708531+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:20.708531+0000","last_peered":"2024-09-15T12:27:20.708531+0000","last_clean":"2024-09-15T12:27:20.708531+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:20.708531+0000","last_undegraded":"2024-09-15T12:27:20.708531+0000","last_fullsized":"2024-09-15T12:27:20.708531+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:26:58.161474+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":26804224,"num_objects":409,"num_object_clones":0,"num_object_copies":1227,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":409,"num_write_kb":26176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3940352,"data_stored":63045632,"data_compressed":2992606,"data_compressed_allocated":3940352,"data_compressed_original":63045632,"omap_allocated":0,"internal_metadata":0},"log_size":409,"ondisk_log_size":409,"up":24,"acting":24,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36540,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":35904,"kb_avail":94335300,"statfs":{"total":96636764160,"available":96599347200,"internally_reserved":0,"allocated":634880,"data_stored":2027012,"data_compressed":89141,"data_compressed_allocated":118784,"data_compressed_original":1818624,"omap_allocated":0,"internal_metadata":36765696},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266717,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36260,"kb_used_data":3988,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":4083712,"data_stored":57142807,"data_compressed":2705416,"data_compressed_allocated":3563520,"data_compressed_original":56934400,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757397,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43088,"kb_used_data":3456,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94328752,"statfs":{"total":96636764160,"available":96592642048,"internally_reserved":0,"allocated":3538944,"data_stored":48936674,"data_compressed":2317376,"data_compressed_allocated":3051520,"data_compressed_original":48742400,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3559424,"data_stored":56950784,"data_compressed":2703359,"data_compressed_allocated":3559424,"data_compressed_original":56950784,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":6094848,"data_compressed":289247,"data_compressed_allocated":380928,"data_compressed_original":6094848,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:27:36.785 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 16 16 1000 984 3.84326 0 - 0.0715028 2024-09-15T12:27:37.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 17 16 1000 984 3.61719 0 - 0.0715028 2024-09-15T12:27:38.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 18 16 1000 984 3.41622 0 - 0.0715028 2024-09-15T12:27:39.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 19 16 1000 984 3.23642 0 - 0.0715028 2024-09-15T12:27:40.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:27:40.785484+0000 min lat: 0.0190733 max lat: 0.163143 avg lat: 0.0715028 2024-09-15T12:27:40.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:27:40.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 20 16 1000 984 3.0746 0 - 0.0715028 2024-09-15T12:27:41.566 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [3, 0, 2] 2024-09-15T12:27:41.566 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:27:41.566 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:27:41.567 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:27:41.570 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:27:41.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:27:41.681 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:27:41.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:41.685 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:27:41.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 21 16 1000 984 2.92818 0 - 0.0715028 2024-09-15T12:27:42.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 22 16 1000 984 2.79509 0 - 0.0715028 2024-09-15T12:27:43.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 23 16 1000 984 2.67357 0 - 0.0715028 2024-09-15T12:27:44.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 24 16 1000 984 2.56217 0 - 0.0715028 2024-09-15T12:27:45.786 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 25 16 1000 984 2.45969 0 - 0.0715028 2024-09-15T12:27:46.686 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:27:46.786 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:27:46.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 26 16 1000 984 2.36509 0 - 0.0715028 2024-09-15T12:27:46.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:27:46.790 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:27:47.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 27 16 1000 984 2.27749 0 - 0.0715028 2024-09-15T12:27:48.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 28 16 1000 984 2.19615 0 - 0.0715028 2024-09-15T12:27:49.686 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:27:49.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 29 16 1000 984 2.12043 0 - 0.0715028 2024-09-15T12:27:49.810 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:27:49.810 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:27:50.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 30 16 1000 984 2.04975 0 - 0.0715028 2024-09-15T12:27:51.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 31 16 1000 984 1.98363 0 - 0.0715028 2024-09-15T12:27:51.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:27:51.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "peering_event", 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 135, 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "PeeringEvent": { 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0", 2024-09-15T12:27:51.968 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "5.8", 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "sent": 62, 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "requested": 62, 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "evt": "epoch_sent: 62 epoch_requested: 62 MInfoRec from 0 info: 5.8( v 59'66 (0'0,59'66] local-lis/les=61/62 n=66 ec=57/57 lis/c=61/57 les/c/f=62/58/0 sis=61) pg_lease(ru 214.102844238s ub 214.233154297s int 16.000000000s)", 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.946246+0000" 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.946263+0000" 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.969 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.946662+0000" 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.948975+0000" 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.953964+0000", 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 2, 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 130 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.972 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:27:51.973 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.974 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:27:51.975 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "peering_event", 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 136, 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "PeeringEvent": { 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0", 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "5.8", 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "sent": 62, 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "requested": 62, 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "evt": "epoch_sent: 62 epoch_requested: 62 MLease epoch 62 from osd.0 pg_lease(ru 214.102844238s ub 214.233154297s int 16.000000000s)", 2024-09-15T12:27:51.976 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.951608+0000" 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.951618+0000" 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.977 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.953605+0000" 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.953778+0000" 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.954175+0000", 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:27:51.979 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PerShardPipeline::create_or_wait_pg", 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 2, 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 130 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:51.980 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.982 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.983 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "background_recovery_sub", 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 130, 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": {} 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 54, 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.828993+0000" 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.829009+0000" 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.830065+0000" 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.831020+0000" 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.833155+0000" 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.833738+0000" 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.988 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.989 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.834531+0000" 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.890868+0000" 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.903694+0000" 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.995 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 56, 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.834583+0000" 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.834595+0000" 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.835036+0000" 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.835410+0000" 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.836208+0000" 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:51.999 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.836733+0000" 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.891403+0000" 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.001 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.895241+0000" 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.917654+0000" 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.007 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 59, 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.838731+0000" 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.838745+0000" 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.838927+0000" 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:52.009 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.839159+0000" 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.840086+0000" 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.011 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.892801+0000" 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.895409+0000" 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.013 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.899493+0000" 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.910405+0000" 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.918857+0000" 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.017 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 61, 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.840003+0000" 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.019 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.840012+0000" 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.840157+0000" 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.840904+0000" 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.021 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.841591+0000" 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.896124+0000" 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.899672+0000" 2024-09-15T12:27:52.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.901779+0000" 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.027 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.915699+0000" 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.939964+0000" 2024-09-15T12:27:52.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.030 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "peering_event", 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 134, 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "PeeringEvent": { 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "0", 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "5.2", 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "sent": 62, 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "requested": 62, 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "evt": "epoch_sent: 62 epoch_requested: 62 RequestRecoveryPrio: priority 190", 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.927928+0000" 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.031 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.927940+0000" 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.930140+0000" 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.032 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.930894+0000" 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.938395+0000" 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.938672+0000" 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.035 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.944919+0000" 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.036 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "peering_event", 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 33554474, 2024-09-15T12:27:52.037 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "PeeringEvent": { 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "1", 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "5.2", 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "sent": 62, 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "requested": 62, 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "evt": "epoch_sent: 62 epoch_requested: 62 RemoteRecoveryReserved", 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "events": [ 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.040 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 51, 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.822428+0000" 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.822454+0000" 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.041 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.822735+0000" 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:27:52.042 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.823180+0000" 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.043 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.823723+0000" 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.044 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.891834+0000" 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.824144+0000" 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.824348+0000" 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:50.824453+0000" 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.048 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:27:51.890055+0000" 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.049 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.051 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:27:52.052 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:27:52.054 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T12:27:52.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:51.997+0000 7f89b767f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:52.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:52.004+0000 7f89b767f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:52.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:52.203+0000 7f74ba548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:52.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:52.222+0000 7f74ba548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:52.701 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T12:27:52.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T12:27:52.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 32 16 1168 1152 2.24973 0.388889 0.0560107 0.444108 2024-09-15T12:27:52.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:52.864+0000 7f3d89551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:52.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:52.884+0000 7f3d89551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:53.707 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T12:27:53.730 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T12:27:53.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 33 16 1382 1366 2.58681 13.375 0.133505 0.385946 2024-09-15T12:27:53.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:53.869+0000 7efd70d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:53.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:53.886+0000 7efd70d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:54.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:54.595+0000 7fdbc6991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:54.603+0000 7fdbc6991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:54.712 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T12:27:54.736 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T12:27:54.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 34 16 1591 1575 2.89488 13.0625 0.113086 0.34477 2024-09-15T12:27:54.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:54.886+0000 7f8262351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:54.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:54.901+0000 7f8262351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:55.717 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T12:27:55.739 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T12:27:55.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:27:55.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 35 16 1805 1789 3.19427 13.375 0.0753802 0.312553 2024-09-15T12:27:55.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:55.888+0000 7fe625991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:55.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:55.904+0000 7fe625991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:56.242 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:56.242 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:27:56.263 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T12:27:56.081280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:27:56.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:27:56.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.432+0000 7f460a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:56.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.443+0000 7f460a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:56.732 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:56.736 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:27:56.751 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T12:27:56.081280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:27:56.753 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:27:56.787 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 36 16 2018 2002 3.47529 13.3125 0.0687803 0.287384 2024-09-15T12:27:56.821 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:27:56.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.901+0000 7f5ec525c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:56.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.910+0000 7f5ec525c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:56.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.985+0000 7ff3852d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:57.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:56.999+0000 7ff3852d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:57.367 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:57.368 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":363,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":66,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":1216,"data_bytes":80085555,"bytes_used":147554304,"bytes_avail":386399502336,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":65522,"recovering_keys_per_sec":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"write_bytes_sec":4439153,"read_op_per_sec":0,"write_op_per_sec":67},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:27:57.392 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T12:27:57.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:27:57.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:57.545+0000 7f84b8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:57.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:57.558+0000 7f84b8d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:57.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 37 16 2221 2205 3.72423 12.6875 0.0605599 0.268251 2024-09-15T12:27:57.897 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:27:57.898 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:27:57.923 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T12:27:56.081280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000820"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:27:58.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 38 16 2420 2404 3.95349 12.4375 0.113737 0.252505 2024-09-15T12:27:59.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:27:59.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:59.455+0000 7efccb2e3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:59.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:27:59.462+0000 7efccb2e3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:27:59.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 39 16 2622 2606 4.1758 12.625 0.0518652 0.239211 2024-09-15T12:28:00.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:28:00.787451+0000 min lat: 0.0189923 max lat: 27.0203 avg lat: 0.226899 2024-09-15T12:28:00.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:28:00.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 40 16 2833 2817 4.40106 13.1875 0.0991341 0.226899 2024-09-15T12:28:00.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:01.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.074+0000 7ff3932fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:01.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.085+0000 7ff3932fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:01.389 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:01.390 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:01.409 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-15T12:28:00.082153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001267"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:01.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:28:01.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.564+0000 7f96b42fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:01.577 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.574+0000 7f96b42fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:01.757 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:01.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 41 16 3045 3029 4.61685 13.25 0.0920925 0.216332 2024-09-15T12:28:01.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.917+0000 7fbc55897640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:01.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:01.929+0000 7fbc55897640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:02.044 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:02.044 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":368,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":67,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":1216,"data_bytes":80085555,"bytes_used":147554304,"bytes_avail":386399502336,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":43681,"recovering_keys_per_sec":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"write_bytes_sec":2959417,"read_op_per_sec":0,"write_op_per_sec":45},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:28:02.065 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T12:28:02.066 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:02.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:02.263+0000 7f380d551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:02.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:02.278+0000 7f380d551640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:02.593 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:02.594 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:02.616 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-15T12:28:02.082578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17760256,"num_objects":271,"num_object_clones":0,"num_object_copies":813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":271,"num_write_kb":17344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.370943"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:02.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 42 16 3251 3235 4.81344 12.875 0.0698002 0.207493 2024-09-15T12:28:03.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 43 16 3463 3447 5.00961 13.25 0.0276437 0.199281 2024-09-15T12:28:04.226 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:04.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:04.381+0000 7f4375b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:04.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:04.389+0000 7f4375b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:04.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 44 16 3668 3652 5.18692 12.8125 0.0752898 0.192562 2024-09-15T12:28:05.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:05.785 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:05.777+0000 7f9532ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:05.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 45 16 3891 3875 5.38134 13.9375 0.0449537 0.185695 2024-09-15T12:28:05.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:05.791+0000 7f9532ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:06.102 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:06.125 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-15T12:28:06.083399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001677"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:06.127 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:28:06.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.285+0000 7f9927591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.295+0000 7f9927591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:06.749 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:06.749 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":373,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":68,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":1216,"data_bytes":80085555,"bytes_used":147554304,"bytes_avail":386399502336,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:28:06.767 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:28:06.768 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:06.788 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 46 16 4092 4076 5.53743 12.5625 0.101485 0.180454 2024-09-15T12:28:06.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.883+0000 7fc4684a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.891+0000 7fc4684a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.962+0000 7f85bb85c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:06.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:06.976+0000 7f85bb85c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:07.304 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:07.304 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:07.323 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-15T12:28:06.083399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001677"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:07.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 47 16 4299 4283 5.69485 12.9375 0.0742561 0.175487 2024-09-15T12:28:08.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 48 16 4491 4475 5.82618 12 0.0681742 0.171398 2024-09-15T12:28:09.206 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:09.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:09.353+0000 7f15290fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:09.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:09.362+0000 7f15290fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:09.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 49 16 4696 4680 5.96873 12.8125 0.0503359 0.167328 2024-09-15T12:28:10.325 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:10.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:10.471+0000 7f717a711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:10.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:10.486+0000 7f717a711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:10.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 50 16 4911 4895 6.11808 13.4375 0.108594 0.163341 2024-09-15T12:28:10.846 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:10.847 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:10.868 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T12:28:10.084185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001607"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:10.869 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:28:11.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.019+0000 7f308f591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.030+0000 7f308f591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.487 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:11.487 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":69,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":1216,"data_bytes":80085555,"bytes_used":147554304,"bytes_avail":386399502336,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:28:11.509 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:28:11.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:11.590 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:11.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.653+0000 7f9334d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.665+0000 7f9334d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.748+0000 7f1bb98ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:11.757+0000 7f1bb98ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:11.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 51 16 5097 5081 6.22604 11.625 0.12233 0.160499 2024-09-15T12:28:11.971 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:11.972 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:11.994 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T12:28:10.084185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80085555,"num_objects":1216,"num_object_clones":0,"num_object_copies":3648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1292,"num_write_kb":78740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1268,"ondisk_log_size":1268,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":144096,"kb_used_data":16352,"kb_used_omap":0,"kb_used_meta":127680,"kb_avail":377343264,"statfs":{"total":386547056640,"available":386399502336,"internally_reserved":0,"allocated":16744448,"data_stored":235072572,"data_compressed":11128929,"data_compressed_allocated":14655488,"data_compressed_original":234242048,"omap_allocated":0,"internal_metadata":130744320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001607"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"59'55","reported_seq":67,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.091071+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:27:25.091071+0000","last_peered":"2024-09-15T12:27:25.091071+0000","last_clean":"2024-09-15T12:27:25.091071+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:27:25.091071+0000","last_undegraded":"2024-09-15T12:27:25.091071+0000","last_fullsized":"2024-09-15T12:27:25.091071+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3604480,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":3520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"59'56","reported_seq":68,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.978730+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:27:24.978730+0000","last_peered":"2024-09-15T12:27:24.978730+0000","last_clean":"2024-09-15T12:27:24.978730+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:27:24.978730+0000","last_undegraded":"2024-09-15T12:27:24.978730+0000","last_fullsized":"2024-09-15T12:27:24.978730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56,"num_write_kb":3584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"59'68","reported_seq":80,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.058424+0000","last_change":"2024-09-15T12:27:19.852425+0000","last_active":"2024-09-15T12:27:25.058424+0000","last_peered":"2024-09-15T12:27:25.058424+0000","last_clean":"2024-09-15T12:27:25.058424+0000","last_became_active":"2024-09-15T12:27:19.841556+0000","last_became_peered":"2024-09-15T12:27:19.841556+0000","last_unstale":"2024-09-15T12:27:25.058424+0000","last_undegraded":"2024-09-15T12:27:25.058424+0000","last_fullsized":"2024-09-15T12:27:25.058424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4456448,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":4352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"59'66","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.222715+0000","last_change":"2024-09-15T12:27:19.918292+0000","last_active":"2024-09-15T12:27:25.222715+0000","last_peered":"2024-09-15T12:27:25.222715+0000","last_clean":"2024-09-15T12:27:25.222715+0000","last_became_active":"2024-09-15T12:27:19.898000+0000","last_became_peered":"2024-09-15T12:27:19.898000+0000","last_unstale":"2024-09-15T12:27:25.222715+0000","last_undegraded":"2024-09-15T12:27:25.222715+0000","last_fullsized":"2024-09-15T12:27:25.222715+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"59'60","reported_seq":72,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:24.998504+0000","last_change":"2024-09-15T12:27:19.908835+0000","last_active":"2024-09-15T12:27:24.998504+0000","last_peered":"2024-09-15T12:27:24.998504+0000","last_clean":"2024-09-15T12:27:24.998504+0000","last_became_active":"2024-09-15T12:27:19.877929+0000","last_became_peered":"2024-09-15T12:27:19.877929+0000","last_unstale":"2024-09-15T12:27:24.998504+0000","last_undegraded":"2024-09-15T12:27:24.998504+0000","last_fullsized":"2024-09-15T12:27:24.998504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":141,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.714609+0000","last_change":"2024-09-15T12:26:58.175326+0000","last_active":"2024-09-15T12:27:20.714609+0000","last_peered":"2024-09-15T12:27:20.714609+0000","last_clean":"2024-09-15T12:27:20.714609+0000","last_became_active":"2024-09-15T12:26:57.294715+0000","last_became_peered":"2024-09-15T12:26:57.294715+0000","last_unstale":"2024-09-15T12:27:20.714609+0000","last_undegraded":"2024-09-15T12:27:20.714609+0000","last_fullsized":"2024-09-15T12:27:20.714609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":143,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:20.706416+0000","last_change":"2024-09-15T12:26:58.177380+0000","last_active":"2024-09-15T12:27:20.706416+0000","last_peered":"2024-09-15T12:27:20.706416+0000","last_clean":"2024-09-15T12:27:20.706416+0000","last_became_active":"2024-09-15T12:26:57.291838+0000","last_became_peered":"2024-09-15T12:26:57.291838+0000","last_unstale":"2024-09-15T12:27:20.706416+0000","last_undegraded":"2024-09-15T12:27:20.706416+0000","last_fullsized":"2024-09-15T12:27:20.706416+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"59'59","reported_seq":71,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.190156+0000","last_change":"2024-09-15T12:27:19.822639+0000","last_active":"2024-09-15T12:27:25.190156+0000","last_peered":"2024-09-15T12:27:25.190156+0000","last_clean":"2024-09-15T12:27:25.190156+0000","last_became_active":"2024-09-15T12:27:19.801035+0000","last_became_peered":"2024-09-15T12:27:19.801035+0000","last_unstale":"2024-09-15T12:27:25.190156+0000","last_undegraded":"2024-09-15T12:27:25.190156+0000","last_fullsized":"2024-09-15T12:27:25.190156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3866624,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":3776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"59'54","reported_seq":66,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T12:27:25.103659+0000","last_change":"2024-09-15T12:27:19.884714+0000","last_active":"2024-09-15T12:27:25.103659+0000","last_peered":"2024-09-15T12:27:25.103659+0000","last_clean":"2024-09-15T12:27:25.103659+0000","last_became_active":"2024-09-15T12:27:19.863317+0000","last_became_peered":"2024-09-15T12:27:19.863317+0000","last_unstale":"2024-09-15T12:27:25.103659+0000","last_undegraded":"2024-09-15T12:27:25.103659+0000","last_fullsized":"2024-09-15T12:27:25.103659+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3538944,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":3456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":79495168,"num_objects":1213,"num_object_clones":0,"num_object_copies":3639,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1213,"num_write_kb":77632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12201984,"data_stored":195231744,"data_compressed":9268033,"data_compressed_allocated":12201984,"data_compressed_original":195231744,"omap_allocated":0,"internal_metadata":0},"log_size":1213,"ondisk_log_size":1213,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757427,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":30860,"kb_used_data":3708,"kb_used_omap":0,"kb_used_meta":27136,"kb_avail":94340980,"statfs":{"total":96636764160,"available":96605163520,"internally_reserved":0,"allocated":3796992,"data_stored":52555287,"data_compressed":2487664,"data_compressed_allocated":3276800,"data_compressed_original":52346880,"omap_allocated":0,"internal_metadata":27787264},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":51773440,"data_compressed":2457608,"data_compressed_allocated":3235840,"data_compressed_original":51773440,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:12.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 52 16 5292 5276 6.34066 12.1875 0.0345408 0.157527 2024-09-15T12:28:13.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 53 16 5507 5491 6.47453 13.4375 0.0395547 0.154309 2024-09-15T12:28:14.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:14.222+0000 7f4d7145c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:14.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:14.231+0000 7f4d7145c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:14.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 54 16 5711 5695 6.59072 12.75 0.0897672 0.15158 2024-09-15T12:28:14.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:15.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:15.150+0000 7f6e8ca9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:15.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:15.165+0000 7f6e8ca9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:15.462 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:15.462 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:15.486 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-15T12:28:14.085102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190906931,"num_objects":2907,"num_object_clones":0,"num_object_copies":8721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2983,"num_write_kb":186964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2959,"ondisk_log_size":2959,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":168932,"kb_used_data":28708,"kb_used_omap":0,"kb_used_meta":140160,"kb_avail":377318428,"statfs":{"total":386547056640,"available":386374070272,"internally_reserved":0,"allocated":29396992,"data_stored":436236170,"data_compressed":20679653,"data_compressed_allocated":27226112,"data_compressed_original":435372032,"omap_allocated":0,"internal_metadata":143523840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110821376,"num_objects":1691,"num_object_clones":0,"num_object_copies":5073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":108224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001702"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":190316544,"num_objects":2904,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":185856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25219072,"data_stored":403505152,"data_compressed":19157965,"data_compressed_allocated":25219072,"data_compressed_original":403505152,"omap_allocated":0,"internal_metadata":0},"log_size":2904,"ondisk_log_size":2904,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:15.488 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:28:15.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:15.646+0000 7fa643f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:15.658+0000 7fa643f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:15.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 55 16 5923 5907 6.71177 13.25 0.0805341 0.148869 2024-09-15T12:28:16.074 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:16.074 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":70,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2907,"data_bytes":190906931,"bytes_used":172986368,"bytes_avail":386374070272,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":57331,"recovering_keys_per_sec":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"write_bytes_sec":13849725,"read_op_per_sec":0,"write_op_per_sec":211},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:28:16.094 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T12:28:16.094 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:16.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:16.255+0000 7f5b1ecd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:16.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:16.265+0000 7f5b1ecd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:16.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:16.550 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:16.551 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:16.566 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-15T12:28:16.085544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190906931,"num_objects":2907,"num_object_clones":0,"num_object_copies":8721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2983,"num_write_kb":186964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2959,"ondisk_log_size":2959,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":168932,"kb_used_data":28708,"kb_used_omap":0,"kb_used_meta":140160,"kb_avail":377318428,"statfs":{"total":386547056640,"available":386374070272,"internally_reserved":0,"allocated":29396992,"data_stored":436236170,"data_compressed":20679653,"data_compressed_allocated":27226112,"data_compressed_original":435372032,"omap_allocated":0,"internal_metadata":143523840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110821376,"num_objects":1691,"num_object_clones":0,"num_object_copies":5073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":108224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001785"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":190316544,"num_objects":2904,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":185856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25219072,"data_stored":403505152,"data_compressed":19157965,"data_compressed_allocated":25219072,"data_compressed_original":403505152,"omap_allocated":0,"internal_metadata":0},"log_size":2904,"ondisk_log_size":2904,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:16.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:16.642+0000 7f665925a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:16.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:16.654+0000 7f665925a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:16.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 56 16 6109 6093 6.79949 11.625 0.122412 0.146903 2024-09-15T12:28:17.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 57 16 6321 6305 6.91263 13.25 0.0775935 0.144576 2024-09-15T12:28:18.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 58 16 6513 6497 7.00033 12 0.115237 0.142731 2024-09-15T12:28:18.956 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:19.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:19.120+0000 7fc721151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:19.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:19.131+0000 7fc721151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:19.569 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:19.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:19.724+0000 7f5ea1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:19.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:19.743+0000 7f5ea1d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:19.789 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 59 16 6732 6716 7.11364 13.6875 0.0465153 0.140475 2024-09-15T12:28:20.089 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:20.089 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:20.110 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-15T12:28:18.687209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190906931,"num_objects":2907,"num_object_clones":0,"num_object_copies":8721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2983,"num_write_kb":186964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2959,"ondisk_log_size":2959,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":168932,"kb_used_data":28708,"kb_used_omap":0,"kb_used_meta":140160,"kb_avail":377318428,"statfs":{"total":386547056640,"available":386374070272,"internally_reserved":0,"allocated":29396992,"data_stored":436236170,"data_compressed":20679653,"data_compressed_allocated":27226112,"data_compressed_original":435372032,"omap_allocated":0,"internal_metadata":143523840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110821376,"num_objects":1691,"num_object_clones":0,"num_object_copies":5073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":108224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":458752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.603023"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":190316544,"num_objects":2904,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":185856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25219072,"data_stored":403505152,"data_compressed":19157965,"data_compressed_allocated":25219072,"data_compressed_original":403505152,"omap_allocated":0,"internal_metadata":0},"log_size":2904,"ondisk_log_size":2904,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:20.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:28:20.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:20.270+0000 7f9ae3f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:20.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:20.284+0000 7f9ae3f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:20.715 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:20.715 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive","count":1},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":387,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":71,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403270,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"stale+active+clean","count":3},{"state_name":"unknown","count":1},{"state_name":"peering","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2907,"data_bytes":190906931,"bytes_used":172986368,"bytes_avail":386374070272,"bytes_total":386547056640,"unknown_pgs_ratio":0.039999999105930328,"inactive_pgs_ratio":0.039999999105930328},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:28:20.740 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:28:20.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:20.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:28:20.789285+0000 min lat: 0.0165234 max lat: 27.0203 avg lat: 0.138725 2024-09-15T12:28:20.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:28:20.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 60 16 6932 6916 7.2034 12.5 0.0833019 0.138725 2024-09-15T12:28:20.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:20.908+0000 7f513d791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:20.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:20.919+0000 7f513d791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:21.275 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:21.275 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:21.292 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-15T12:28:20.087429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":190906931,"num_objects":2907,"num_object_clones":0,"num_object_copies":8721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2983,"num_write_kb":186964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2959,"ondisk_log_size":2959,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":168932,"kb_used_data":28708,"kb_used_omap":0,"kb_used_meta":140160,"kb_avail":377318428,"statfs":{"total":386547056640,"available":386374070272,"internally_reserved":0,"allocated":29396992,"data_stored":436236170,"data_compressed":20679653,"data_compressed_allocated":27226112,"data_compressed_original":435372032,"omap_allocated":0,"internal_metadata":143523840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002826"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"59'74","reported_seq":88,"reported_epoch":61,"state":"peering","last_fresh":"2024-09-15T12:27:49.723503+0000","last_change":"2024-09-15T12:27:49.723503+0000","last_active":"2024-09-15T12:27:49.723503+0000","last_peered":"2024-09-15T12:27:25.014749+0000","last_clean":"2024-09-15T12:27:25.014749+0000","last_became_active":"2024-09-15T12:27:19.816192+0000","last_became_peered":"2024-09-15T12:27:19.816192+0000","last_unstale":"2024-09-15T12:27:49.723503+0000","last_undegraded":"2024-09-15T12:27:49.723503+0000","last_fullsized":"2024-09-15T12:27:49.723503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4849664,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":74,"num_write_kb":4736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:27:18.686229+0000","last_change":"2024-09-15T12:27:18.686229+0000","last_active":"2024-09-15T12:27:18.686229+0000","last_peered":"2024-09-15T12:27:18.686229+0000","last_clean":"2024-09-15T12:27:18.686229+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:27:18.686229+0000","last_undegraded":"2024-09-15T12:27:18.686229+0000","last_fullsized":"2024-09-15T12:27:18.686229+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":180,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.703475+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:27:50.703475+0000","last_peered":"2024-09-15T12:27:50.703475+0000","last_clean":"2024-09-15T12:27:50.703475+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:27:50.703475+0000","last_undegraded":"2024-09-15T12:27:50.703475+0000","last_fullsized":"2024-09-15T12:27:50.703475+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.166967+0000","last_change":"2024-09-15T12:26:58.174285+0000","last_active":"2024-09-15T12:26:58.166967+0000","last_peered":"2024-09-15T12:26:58.166967+0000","last_clean":"2024-09-15T12:26:58.166967+0000","last_became_active":"2024-09-15T12:26:57.305406+0000","last_became_peered":"2024-09-15T12:26:57.305406+0000","last_unstale":"2024-09-15T12:27:49.683089+0000","last_undegraded":"2024-09-15T12:26:58.166967+0000","last_fullsized":"2024-09-15T12:26:58.166967+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.178043+0000","last_change":"2024-09-15T12:26:58.185458+0000","last_active":"2024-09-15T12:26:58.178043+0000","last_peered":"2024-09-15T12:26:58.178043+0000","last_clean":"2024-09-15T12:26:58.178043+0000","last_became_active":"2024-09-15T12:26:57.261581+0000","last_became_peered":"2024-09-15T12:26:57.261581+0000","last_unstale":"2024-09-15T12:27:49.683092+0000","last_undegraded":"2024-09-15T12:26:58.178043+0000","last_fullsized":"2024-09-15T12:26:58.178043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":117,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.720659+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:27:50.720659+0000","last_peered":"2024-09-15T12:27:50.720659+0000","last_clean":"2024-09-15T12:27:50.720659+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:27:50.720659+0000","last_undegraded":"2024-09-15T12:27:50.720659+0000","last_fullsized":"2024-09-15T12:27:50.720659+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"59'57","reported_seq":75,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T12:27:50.713013+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:27:50.713013+0000","last_peered":"2024-09-15T12:27:50.713013+0000","last_clean":"2024-09-15T12:27:50.713013+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:27:50.713013+0000","last_undegraded":"2024-09-15T12:27:50.713013+0000","last_fullsized":"2024-09-15T12:27:50.713013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3735552,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":3648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":55,"state":"stale+active+clean","last_fresh":"2024-09-15T12:26:58.161474+0000","last_change":"2024-09-15T12:26:58.167234+0000","last_active":"2024-09-15T12:26:58.161474+0000","last_peered":"2024-09-15T12:26:58.161474+0000","last_clean":"2024-09-15T12:26:58.161474+0000","last_became_active":"2024-09-15T12:26:57.265994+0000","last_became_peered":"2024-09-15T12:26:57.265994+0000","last_unstale":"2024-09-15T12:27:49.683096+0000","last_undegraded":"2024-09-15T12:26:58.161474+0000","last_fullsized":"2024-09-15T12:26:58.161474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":190316544,"num_objects":2904,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":185856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25219072,"data_stored":403505152,"data_compressed":19157965,"data_compressed_allocated":25219072,"data_compressed_original":403505152,"omap_allocated":0,"internal_metadata":0},"log_size":2904,"ondisk_log_size":2904,"up":45,"acting":45,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":43416,"kb_used_data":3336,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":94328424,"statfs":{"total":96636764160,"available":96592306176,"internally_reserved":0,"allocated":3416064,"data_stored":46271242,"data_compressed":2189014,"data_compressed_allocated":2883584,"data_compressed_original":46055424,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31028,"kb_used_data":2788,"kb_used_omap":0,"kb_used_meta":28224,"kb_avail":94340812,"statfs":{"total":96636764160,"available":96604991488,"internally_reserved":0,"allocated":2854912,"data_stored":38384834,"data_compressed":1814296,"data_compressed_allocated":2387968,"data_compressed_original":38207488,"omap_allocated":0,"internal_metadata":28901376},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2842624,"data_stored":45481984,"data_compressed":2158958,"data_compressed_allocated":2842624,"data_compressed_original":45481984,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:21.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:21.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:21.643+0000 7f0cab191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:21.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:21.651+0000 7f0cab191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:21.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 61 16 7140 7124 7.2984 13 0.0789796 0.136931 2024-09-15T12:28:22.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 62 16 7336 7320 7.37825 12.25 0.0354309 0.13539 2024-09-15T12:28:23.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 63 16 7541 7525 7.46449 12.8125 0.0834024 0.133795 2024-09-15T12:28:23.982 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:24.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:24.139+0000 7fe19108e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:24.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:24.148+0000 7fe19108e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:24.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:24.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:24.457+0000 7f217ce6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:24.476+0000 7f217ce6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:24.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 64 16 7731 7715 7.53339 11.875 0.1201 0.132621 2024-09-15T12:28:24.800 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:24.801 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:24.819 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-15T12:28:24.088226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":269681203,"num_objects":4109,"num_object_clones":0,"num_object_copies":12327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":4185,"num_write_kb":263892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4162,"ondisk_log_size":4162,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":222636,"kb_used_data":59820,"kb_used_omap":0,"kb_used_meta":162752,"kb_avail":377264724,"statfs":{"total":386547056640,"available":386319077376,"internally_reserved":0,"allocated":61255680,"data_stored":942654297,"data_compressed":44722254,"data_compressed_allocated":58871808,"data_compressed_original":941703168,"omap_allocated":0,"internal_metadata":166658048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":78774272,"num_objects":1202,"num_object_clones":0,"num_object_copies":3606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1202,"num_write_kb":76928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002681"},"pg_stats":[{"pgid":"5.f","version":"65'115","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.249583+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:27:55.249583+0000","last_peered":"2024-09-15T12:27:55.249583+0000","last_clean":"2024-09-15T12:27:55.249583+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:27:55.249583+0000","last_undegraded":"2024-09-15T12:27:55.249583+0000","last_fullsized":"2024-09-15T12:27:55.249583+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7536640,"num_objects":115,"num_object_clones":0,"num_object_copies":345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":7360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:21.367873+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"65'101","reported_seq":131,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.031327+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:27:55.031327+0000","last_peered":"2024-09-15T12:27:55.031327+0000","last_clean":"2024-09-15T12:27:55.031327+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:27:55.031327+0000","last_undegraded":"2024-09-15T12:27:55.031327+0000","last_fullsized":"2024-09-15T12:27:55.031327+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6619136,"num_objects":101,"num_object_clones":0,"num_object_copies":303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":6464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"71'403","reported_seq":450,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:20.415964+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:28:20.415964+0000","last_peered":"2024-09-15T12:28:20.415964+0000","last_clean":"2024-09-15T12:28:20.415964+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:28:20.415964+0000","last_undegraded":"2024-09-15T12:28:20.415964+0000","last_fullsized":"2024-09-15T12:28:20.415964+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26345472,"num_objects":402,"num_object_clones":0,"num_object_copies":1206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":402,"num_write_kb":25728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"65'114","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.174910+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:27:55.174910+0000","last_peered":"2024-09-15T12:27:55.174910+0000","last_clean":"2024-09-15T12:27:55.174910+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:27:55.174910+0000","last_undegraded":"2024-09-15T12:27:55.174910+0000","last_fullsized":"2024-09-15T12:27:55.174910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7471104,"num_objects":114,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":114,"num_write_kb":7296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:18.711700+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":143,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.730356+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:28:18.730356+0000","last_peered":"2024-09-15T12:28:18.730356+0000","last_clean":"2024-09-15T12:28:18.730356+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:28:18.730356+0000","last_undegraded":"2024-09-15T12:28:18.730356+0000","last_fullsized":"2024-09-15T12:28:18.730356+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"65'108","reported_seq":138,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.072336+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:27:55.072336+0000","last_peered":"2024-09-15T12:27:55.072336+0000","last_clean":"2024-09-15T12:27:55.072336+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:27:55.072336+0000","last_undegraded":"2024-09-15T12:27:55.072336+0000","last_fullsized":"2024-09-15T12:27:55.072336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7077888,"num_objects":108,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.720851+0000","last_change":"2024-09-15T12:27:52.738885+0000","last_active":"2024-09-15T12:28:18.720851+0000","last_peered":"2024-09-15T12:28:18.720851+0000","last_clean":"2024-09-15T12:28:18.720851+0000","last_became_active":"2024-09-15T12:27:51.776000+0000","last_became_peered":"2024-09-15T12:27:51.776000+0000","last_unstale":"2024-09-15T12:28:18.720851+0000","last_undegraded":"2024-09-15T12:28:18.720851+0000","last_fullsized":"2024-09-15T12:28:18.720851+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"65'116","reported_seq":154,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.238076+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:27:55.238076+0000","last_peered":"2024-09-15T12:27:55.238076+0000","last_clean":"2024-09-15T12:27:55.238076+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:27:55.238076+0000","last_undegraded":"2024-09-15T12:27:55.238076+0000","last_fullsized":"2024-09-15T12:27:55.238076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7602176,"num_objects":116,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":116,"num_write_kb":7424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"65'110","reported_seq":148,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:55.048895+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:27:55.048895+0000","last_peered":"2024-09-15T12:27:55.048895+0000","last_clean":"2024-09-15T12:27:55.048895+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:27:55.048895+0000","last_undegraded":"2024-09-15T12:27:55.048895+0000","last_fullsized":"2024-09-15T12:27:55.048895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7208960,"num_objects":110,"num_object_clones":0,"num_object_copies":330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":7040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.772863+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:27:54.772863+0000","last_peered":"2024-09-15T12:27:54.772863+0000","last_clean":"2024-09-15T12:27:54.772863+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:27:54.772863+0000","last_undegraded":"2024-09-15T12:27:54.772863+0000","last_fullsized":"2024-09-15T12:27:54.772863+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:18.720045+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:21.256790+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T12:27:54.770160+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:27:54.770160+0000","last_peered":"2024-09-15T12:27:54.770160+0000","last_clean":"2024-09-15T12:27:54.770160+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:27:54.770160+0000","last_undegraded":"2024-09-15T12:27:54.770160+0000","last_fullsized":"2024-09-15T12:27:54.770160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":143,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.722940+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:28:18.722940+0000","last_peered":"2024-09-15T12:28:18.722940+0000","last_clean":"2024-09-15T12:28:18.722940+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:28:18.722940+0000","last_undegraded":"2024-09-15T12:28:18.722940+0000","last_fullsized":"2024-09-15T12:28:18.722940+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":269090816,"num_objects":4106,"num_object_clones":0,"num_object_copies":12318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4106,"num_write_kb":262784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":59961344,"data_stored":959381504,"data_compressed":45553238,"data_compressed_allocated":59961344,"data_compressed_original":959381504,"omap_allocated":0,"internal_metadata":0},"log_size":4107,"ondisk_log_size":4107,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266747,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":38792,"kb_used_data":6520,"kb_used_omap":0,"kb_used_meta":32256,"kb_avail":94333048,"statfs":{"total":96636764160,"available":96597041152,"internally_reserved":0,"allocated":6676480,"data_stored":97861209,"data_compressed":4637955,"data_compressed_allocated":6107136,"data_compressed_original":97632256,"omap_allocated":0,"internal_metadata":33030144},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50176,"kb_used_data":17392,"kb_used_omap":0,"kb_used_meta":32768,"kb_avail":94321664,"statfs":{"total":96636764160,"available":96585383936,"internally_reserved":0,"allocated":17809408,"data_stored":275613895,"data_compressed":13075913,"data_compressed_allocated":17211392,"data_compressed_original":275382272,"omap_allocated":0,"internal_metadata":33554432},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17690624,"data_stored":283049984,"data_compressed":13440017,"data_compressed_allocated":17690624,"data_compressed_original":283049984,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6123520,"data_stored":97976320,"data_compressed":4651467,"data_compressed_allocated":6123520,"data_compressed_original":97976320,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:24.821 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T12:28:24.821 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:28:24.821 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [3, 0, 2, 1] 2024-09-15T12:28:24.822 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:28:24.822 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:28:24.910 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:28:25.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 65 16 7749 7733 7.43479 1.125 0.0277935 0.132494 2024-09-15T12:28:26.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:26.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:26.585+0000 7efcdca9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:26.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:26.590+0000 7efcdca9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:26.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 66 16 7749 7733 7.32214 0 - 0.132494 2024-09-15T12:28:27.790 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 67 16 7749 7733 7.21285 0 - 0.132494 2024-09-15T12:28:28.791 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 68 16 7749 7733 7.10677 0 - 0.132494 2024-09-15T12:28:29.791 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 69 16 7749 7733 7.00377 0 - 0.132494 2024-09-15T12:28:29.911 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:28:29.911 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:28:29.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:28:30.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:30.027+0000 7ff336991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:30.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:30.035+0000 7ff336991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:30.221 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:28:30.222 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:28:30.234 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T12:28:30.089710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419037747,"num_objects":6388,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":6464,"num_write_kb":409748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6441,"ondisk_log_size":6441,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":247768,"kb_used_data":80792,"kb_used_omap":0,"kb_used_meta":166912,"kb_avail":377239592,"statfs":{"total":386547056640,"available":386293342208,"internally_reserved":0,"allocated":82731008,"data_stored":1285364904,"data_compressed":60994902,"data_compressed_allocated":80289792,"data_compressed_original":1284390912,"omap_allocated":0,"internal_metadata":170917888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":228130816,"num_objects":3481,"num_object_clones":0,"num_object_copies":10443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3481,"num_write_kb":222784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002280"},"pg_stats":[{"pgid":"5.f","version":"72'511","reported_seq":562,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.842554+0000","last_change":"2024-09-15T12:27:19.854792+0000","last_active":"2024-09-15T12:28:24.842554+0000","last_peered":"2024-09-15T12:28:24.842554+0000","last_clean":"2024-09-15T12:28:24.842554+0000","last_became_active":"2024-09-15T12:27:19.830995+0000","last_became_peered":"2024-09-15T12:27:19.830995+0000","last_unstale":"2024-09-15T12:28:24.842554+0000","last_undegraded":"2024-09-15T12:28:24.842554+0000","last_fullsized":"2024-09-15T12:28:24.842554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33488896,"num_objects":511,"num_object_clones":0,"num_object_copies":1533,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":511,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"69'301","reported_seq":343,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.950572+0000","last_change":"2024-09-15T12:27:19.914533+0000","last_active":"2024-09-15T12:28:10.950572+0000","last_peered":"2024-09-15T12:28:10.950572+0000","last_clean":"2024-09-15T12:28:10.950572+0000","last_became_active":"2024-09-15T12:27:19.890418+0000","last_became_peered":"2024-09-15T12:27:19.890418+0000","last_unstale":"2024-09-15T12:28:10.950572+0000","last_undegraded":"2024-09-15T12:28:10.950572+0000","last_fullsized":"2024-09-15T12:28:10.950572+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19726336,"num_objects":301,"num_object_clones":0,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":301,"num_write_kb":19264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"69'290","reported_seq":332,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.866366+0000","last_change":"2024-09-15T12:27:19.904376+0000","last_active":"2024-09-15T12:28:10.866366+0000","last_peered":"2024-09-15T12:28:10.866366+0000","last_clean":"2024-09-15T12:28:10.866366+0000","last_became_active":"2024-09-15T12:27:19.868267+0000","last_became_peered":"2024-09-15T12:27:19.868267+0000","last_unstale":"2024-09-15T12:28:10.866366+0000","last_undegraded":"2024-09-15T12:28:10.866366+0000","last_fullsized":"2024-09-15T12:28:10.866366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19005440,"num_objects":290,"num_object_clones":0,"num_object_copies":870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":290,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:21.367873+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"69'288","reported_seq":331,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.946436+0000","last_change":"2024-09-15T12:27:51.808687+0000","last_active":"2024-09-15T12:28:10.946436+0000","last_peered":"2024-09-15T12:28:10.946436+0000","last_clean":"2024-09-15T12:28:10.946436+0000","last_became_active":"2024-09-15T12:27:51.796708+0000","last_became_peered":"2024-09-15T12:27:51.796708+0000","last_unstale":"2024-09-15T12:28:10.946436+0000","last_undegraded":"2024-09-15T12:28:10.946436+0000","last_fullsized":"2024-09-15T12:28:10.946436+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":18432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"72'460","reported_seq":511,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.702552+0000","last_change":"2024-09-15T12:27:19.888520+0000","last_active":"2024-09-15T12:28:24.702552+0000","last_peered":"2024-09-15T12:28:24.702552+0000","last_clean":"2024-09-15T12:28:24.702552+0000","last_became_active":"2024-09-15T12:27:19.857975+0000","last_became_peered":"2024-09-15T12:27:19.857975+0000","last_unstale":"2024-09-15T12:28:24.702552+0000","last_undegraded":"2024-09-15T12:28:24.702552+0000","last_fullsized":"2024-09-15T12:28:24.702552+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30146560,"num_objects":460,"num_object_clones":0,"num_object_copies":1380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":460,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"71'403","reported_seq":450,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:20.415964+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:28:20.415964+0000","last_peered":"2024-09-15T12:28:20.415964+0000","last_clean":"2024-09-15T12:28:20.415964+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:28:20.415964+0000","last_undegraded":"2024-09-15T12:28:20.415964+0000","last_fullsized":"2024-09-15T12:28:20.415964+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26345472,"num_objects":402,"num_object_clones":0,"num_object_copies":1206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":402,"num_write_kb":25728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"69'324","reported_seq":378,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.865943+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:28:10.865943+0000","last_peered":"2024-09-15T12:28:10.865943+0000","last_clean":"2024-09-15T12:28:10.865943+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:28:10.865943+0000","last_undegraded":"2024-09-15T12:28:10.865943+0000","last_fullsized":"2024-09-15T12:28:10.865943+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":324,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":324,"num_write_kb":20736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"72'494","reported_seq":546,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.688152+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:28:24.688152+0000","last_peered":"2024-09-15T12:28:24.688152+0000","last_clean":"2024-09-15T12:28:24.688152+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:28:24.688152+0000","last_undegraded":"2024-09-15T12:28:24.688152+0000","last_fullsized":"2024-09-15T12:28:24.688152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32374784,"num_objects":494,"num_object_clones":0,"num_object_copies":1482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":494,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:18.711700+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":143,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.730356+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:28:18.730356+0000","last_peered":"2024-09-15T12:28:18.730356+0000","last_clean":"2024-09-15T12:28:18.730356+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:28:18.730356+0000","last_undegraded":"2024-09-15T12:28:18.730356+0000","last_fullsized":"2024-09-15T12:28:18.730356+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"69'312","reported_seq":355,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.962717+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:28:10.962717+0000","last_peered":"2024-09-15T12:28:10.962717+0000","last_clean":"2024-09-15T12:28:10.962717+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:28:10.962717+0000","last_undegraded":"2024-09-15T12:28:10.962717+0000","last_fullsized":"2024-09-15T12:28:10.962717+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20447232,"num_objects":312,"num_object_clones":0,"num_object_copies":936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":312,"num_write_kb":19968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":173,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.704424+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:08.704424+0000","last_peered":"2024-09-15T12:28:08.704424+0000","last_clean":"2024-09-15T12:28:08.704424+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:08.704424+0000","last_undegraded":"2024-09-15T12:28:08.704424+0000","last_fullsized":"2024-09-15T12:28:08.704424+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"72'502","reported_seq":553,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.852079+0000","last_change":"2024-09-15T12:27:19.926344+0000","last_active":"2024-09-15T12:28:24.852079+0000","last_peered":"2024-09-15T12:28:24.852079+0000","last_clean":"2024-09-15T12:28:24.852079+0000","last_became_active":"2024-09-15T12:27:19.921625+0000","last_became_peered":"2024-09-15T12:27:19.921625+0000","last_unstale":"2024-09-15T12:28:24.852079+0000","last_undegraded":"2024-09-15T12:28:24.852079+0000","last_fullsized":"2024-09-15T12:28:24.852079+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":502,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":502,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":130,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.720851+0000","last_change":"2024-09-15T12:27:52.738885+0000","last_active":"2024-09-15T12:28:18.720851+0000","last_peered":"2024-09-15T12:28:18.720851+0000","last_clean":"2024-09-15T12:28:18.720851+0000","last_became_active":"2024-09-15T12:27:51.776000+0000","last_became_peered":"2024-09-15T12:27:51.776000+0000","last_unstale":"2024-09-15T12:28:18.720851+0000","last_undegraded":"2024-09-15T12:28:18.720851+0000","last_fullsized":"2024-09-15T12:28:18.720851+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"72'501","reported_seq":560,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.762268+0000","last_change":"2024-09-15T12:27:51.970313+0000","last_active":"2024-09-15T12:28:24.762268+0000","last_peered":"2024-09-15T12:28:24.762268+0000","last_clean":"2024-09-15T12:28:24.762268+0000","last_became_active":"2024-09-15T12:27:51.803485+0000","last_became_peered":"2024-09-15T12:27:51.803485+0000","last_unstale":"2024-09-15T12:28:24.762268+0000","last_undegraded":"2024-09-15T12:28:24.762268+0000","last_fullsized":"2024-09-15T12:28:24.762268+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32833536,"num_objects":501,"num_object_clones":0,"num_object_copies":1503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":501,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":175,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:08.698499+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:08.698499+0000","last_peered":"2024-09-15T12:28:08.698499+0000","last_clean":"2024-09-15T12:28:08.698499+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:08.698499+0000","last_undegraded":"2024-09-15T12:28:08.698499+0000","last_fullsized":"2024-09-15T12:28:08.698499+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"72'475","reported_seq":534,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:24.730679+0000","last_change":"2024-09-15T12:27:51.946817+0000","last_active":"2024-09-15T12:28:24.730679+0000","last_peered":"2024-09-15T12:28:24.730679+0000","last_clean":"2024-09-15T12:28:24.730679+0000","last_became_active":"2024-09-15T12:27:51.801383+0000","last_became_peered":"2024-09-15T12:27:51.801383+0000","last_unstale":"2024-09-15T12:28:24.730679+0000","last_undegraded":"2024-09-15T12:28:24.730679+0000","last_fullsized":"2024-09-15T12:28:24.730679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":475,"num_object_clones":0,"num_object_copies":1425,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":475,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":172,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:23.737359+0000","last_change":"2024-09-15T12:26:58.196863+0000","last_active":"2024-09-15T12:28:23.737359+0000","last_peered":"2024-09-15T12:28:23.737359+0000","last_clean":"2024-09-15T12:28:23.737359+0000","last_became_active":"2024-09-15T12:26:57.392165+0000","last_became_peered":"2024-09-15T12:26:57.392165+0000","last_unstale":"2024-09-15T12:28:23.737359+0000","last_undegraded":"2024-09-15T12:28:23.737359+0000","last_fullsized":"2024-09-15T12:28:23.737359+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"69'303","reported_seq":353,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.868834+0000","last_change":"2024-09-15T12:27:52.009495+0000","last_active":"2024-09-15T12:28:10.868834+0000","last_peered":"2024-09-15T12:28:10.868834+0000","last_clean":"2024-09-15T12:28:10.868834+0000","last_became_active":"2024-09-15T12:27:51.833813+0000","last_became_peered":"2024-09-15T12:27:51.833813+0000","last_unstale":"2024-09-15T12:28:10.868834+0000","last_undegraded":"2024-09-15T12:28:10.868834+0000","last_fullsized":"2024-09-15T12:28:10.868834+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19857408,"num_objects":303,"num_object_clones":0,"num_object_copies":909,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":303,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:18.720045+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:21.256790+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":172,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T12:28:23.736067+0000","last_change":"2024-09-15T12:26:58.201476+0000","last_active":"2024-09-15T12:28:23.736067+0000","last_peered":"2024-09-15T12:28:23.736067+0000","last_clean":"2024-09-15T12:28:23.736067+0000","last_became_active":"2024-09-15T12:26:57.326823+0000","last_became_peered":"2024-09-15T12:26:57.326823+0000","last_unstale":"2024-09-15T12:28:23.736067+0000","last_undegraded":"2024-09-15T12:28:23.736067+0000","last_fullsized":"2024-09-15T12:28:23.736067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"69'291","reported_seq":339,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-15T12:28:10.807721+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:28:10.807721+0000","last_peered":"2024-09-15T12:28:10.807721+0000","last_clean":"2024-09-15T12:28:10.807721+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:28:10.807721+0000","last_undegraded":"2024-09-15T12:28:10.807721+0000","last_fullsized":"2024-09-15T12:28:10.807721+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":291,"num_write_kb":18624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":143,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-15T12:28:18.722940+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:28:18.722940+0000","last_peered":"2024-09-15T12:28:18.722940+0000","last_clean":"2024-09-15T12:28:18.722940+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:28:18.722940+0000","last_undegraded":"2024-09-15T12:28:18.722940+0000","last_fullsized":"2024-09-15T12:28:18.722940+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":418447360,"num_objects":6385,"num_object_clones":0,"num_object_copies":19155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6385,"num_write_kb":408640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":720896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81719296,"data_stored":1307508736,"data_compressed":62084182,"data_compressed_allocated":81719296,"data_compressed_original":1307508736,"omap_allocated":0,"internal_metadata":0},"log_size":6386,"ondisk_log_size":6386,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266776,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":63924,"kb_used_data":27492,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94307916,"statfs":{"total":96636764160,"available":96571305984,"internally_reserved":0,"allocated":28151808,"data_stored":440571816,"data_compressed":20910603,"data_compressed_allocated":27525120,"data_compressed_original":440320000,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005085,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50176,"kb_used_data":17392,"kb_used_omap":0,"kb_used_meta":32768,"kb_avail":94321664,"statfs":{"total":96636764160,"available":96585383936,"internally_reserved":0,"allocated":17809408,"data_stored":275613895,"data_compressed":13075913,"data_compressed_allocated":17211392,"data_compressed_original":275382272,"omap_allocated":0,"internal_metadata":33554432},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55696,"kb_used_data":16064,"kb_used_omap":0,"kb_used_meta":39616,"kb_avail":94316144,"statfs":{"total":96636764160,"available":96579731456,"internally_reserved":0,"allocated":16449536,"data_stored":253718885,"data_compressed":12038388,"data_compressed_allocated":15847424,"data_compressed_original":253476864,"omap_allocated":0,"internal_metadata":40566784},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16252928,"data_stored":260046848,"data_compressed":12347540,"data_compressed_allocated":16252928,"data_compressed_original":260046848,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17690624,"data_stored":283049984,"data_compressed":13440017,"data_compressed_allocated":17690624,"data_compressed_original":283049984,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":27881472,"data_stored":446103552,"data_compressed":21182411,"data_compressed_allocated":27881472,"data_compressed_original":446103552,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:28:30.791 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 70 16 7749 7733 6.90371 0 - 0.132494 2024-09-15T12:28:31.791 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 71 16 7749 7733 6.80648 0 - 0.132494 2024-09-15T12:28:32.791 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 72 16 7749 7733 6.71194 0 - 0.132494 2024-09-15T12:28:33.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 73 16 7749 7733 6.61999 0 - 0.132494 2024-09-15T12:28:34.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 74 16 7749 7733 6.53053 0 - 0.132494 2024-09-15T12:28:35.236 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:28:35.236 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:28:35.236 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-15T12:28:35.236 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:28:35.237 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:28:35.240 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:28:35.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:28:35.351 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:28:35.354 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:35.354 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:28:35.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 75 16 7749 7733 6.44345 0 - 0.132494 2024-09-15T12:28:36.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 76 16 7749 7733 6.35866 0 - 0.132494 2024-09-15T12:28:37.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 77 16 7749 7733 6.27608 0 - 0.132494 2024-09-15T12:28:38.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 78 16 7749 7733 6.19562 0 - 0.132494 2024-09-15T12:28:39.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 79 16 7749 7733 6.11719 0 - 0.132494 2024-09-15T12:28:40.355 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:28:40.456 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:28:40.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:40.459 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:28:40.792 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:28:40.791883+0000 min lat: 0.0165234 max lat: 27.0203 avg lat: 0.132494 2024-09-15T12:28:40.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:28:40.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 80 16 7749 7733 6.04073 0 - 0.132494 2024-09-15T12:28:41.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 81 16 7749 7733 5.96615 0 - 0.132494 2024-09-15T12:28:42.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 82 16 7749 7733 5.89338 0 - 0.132494 2024-09-15T12:28:43.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 83 16 7749 7733 5.82238 0 - 0.132494 2024-09-15T12:28:44.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 84 16 7749 7733 5.75306 0 - 0.132494 2024-09-15T12:28:45.460 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:28:45.560 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:28:45.563 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:45.563 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:28:45.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 85 16 7749 7733 5.68537 0 - 0.132494 2024-09-15T12:28:46.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 86 16 7749 7733 5.61926 0 - 0.132494 2024-09-15T12:28:47.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 87 16 7749 7733 5.55467 0 - 0.132494 2024-09-15T12:28:48.700 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:28:48.705 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:28:48.705 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:28:48.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 88 16 7749 7733 5.49155 0 - 0.132494 2024-09-15T12:28:49.793 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 89 16 7749 7733 5.42985 0 - 0.132494 2024-09-15T12:28:50.564 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:28:50.669 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:28:50.670 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [] 2024-09-15T12:28:50.670 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:28:50.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:50.794 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 90 16 7749 7733 5.36952 0 - 0.132494 2024-09-15T12:28:50.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:50.858+0000 7f6d16548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:50.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:50.866+0000 7f6d16548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:51.794 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total time run: 90.3055 2024-09-15T12:28:51.794 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total writes made: 7749 2024-09-15T12:28:51.794 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Write size: 65536 2024-09-15T12:28:51.794 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object size: 65536 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Bandwidth (MB/sec): 5.36305 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Bandwidth: 6.35434 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max bandwidth (MB/sec): 14.4375 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min bandwidth (MB/sec): 0 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average IOPS: 85 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev IOPS: 101.671 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max IOPS: 231 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min IOPS: 0 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average Latency(s): 0.186332 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Latency(s): 1.6969 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max latency(s): 27.0203 2024-09-15T12:28:51.795 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min latency(s): 0.0165234 2024-09-15T12:28:51.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T12:28:53.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:53.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:53.245+0000 7f32b4711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:53.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:53.257+0000 7f32b4711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:55.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:28:55.680 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:28:55.680 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:28:55.680 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:28:55.680 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:28:55.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:55.728+0000 7fb19d951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:55.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:55.741+0000 7fb19d951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:55.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:55.848+0000 7f6a5df91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:55.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:55.866+0000 7f6a5df91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.028 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:28:56.029 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:28:56.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:56.033 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:28:56.033 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:28:56.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.181+0000 7f1b3ed91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.199+0000 7f1b3ed91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:28:56.244 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:28:56.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:56.247 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:28:56.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:28:56.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.411+0000 7f251cca3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.429+0000 7f251cca3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:28:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:28:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:28:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:28:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:28:56.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:56.526 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:28:56.526 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:28:56.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.682+0000 7fdb988fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:56.701+0000 7fdb988fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:56.797 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:28:56.797 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:28:56.798 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:28:56.802 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:28:56.802 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:28:58.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:28:58.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:58.287+0000 7f0c97151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:28:58.296 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:28:58.295+0000 7f0c97151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:00.734 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:00.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:00.896+0000 7f35e8cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:00.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:00.905+0000 7f35e8cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:01.802 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:29:01.803 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:29:01.803 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 2, 1, 3] 2024-09-15T12:29:01.803 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:29:01.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:29:01.896 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:29:03.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:03.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:03.371+0000 7fe4fc711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:03.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:03.376+0000 7fe4fc711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:06.896 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:29:06.897 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-15T12:29:06.897 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:29:06.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:29:06.899 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:29:06.900 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:29:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:29:07.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:29:07.010 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:29:12.010 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:29:12.108 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:29:12.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:29:12.111 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:29:15.160 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:29:15.164 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:29:15.164 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:29:17.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:29:17.166 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:17.250 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:29:17.250 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [] 2024-09-15T12:29:17.250 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:29:17.268 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T12:29:17.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:17.322+0000 7f63ea511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:17.332 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:17.330+0000 7f63ea511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:17.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:17.413+0000 7fbcdd2a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:17.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:17.430+0000 7fbcdd2a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:18.177 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T12:29:18.201 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T12:29:18.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:18.372+0000 7f2bfe548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:18.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:18.389+0000 7f2bfe548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:19.184 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T12:29:19.202 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T12:29:19.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:19.343+0000 7ff789f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:19.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:19.363+0000 7ff789f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:19.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:19.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:19.791+0000 7f261b09c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:19.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:19.801+0000 7f261b09c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:20.195 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T12:29:20.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T12:29:20.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:20.383+0000 7fdb74191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:20.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:20.403+0000 7fdb74191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:21.200 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T12:29:21.223 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T12:29:21.224 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:21.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:21.385+0000 7f50bce63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:21.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:21.397+0000 7f50bce63640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:21.700 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:21.701 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:21.719 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-15T12:29:21.199728+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398459467,"num_objects":6075,"num_object_clones":0,"num_object_copies":18225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":9335,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9312,"ondisk_log_size":9312,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":273488,"kb_used_data":75984,"kb_used_omap":0,"kb_used_meta":197440,"kb_avail":377213872,"statfs":{"total":386547056640,"available":386267004928,"internally_reserved":0,"allocated":77807616,"data_stored":1203779573,"data_compressed":57119591,"data_compressed_allocated":75186176,"data_compressed_original":1202733056,"omap_allocated":0,"internal_metadata":202178560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11272192,"num_objects":-172,"num_object_clones":0,"num_object_copies":-516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030022"},"pg_stats":[{"pgid":"5.f","version":"76'572","reported_seq":645,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.143630+0000","last_change":"2024-09-15T12:28:51.086796+0000","last_active":"2024-09-15T12:28:55.143630+0000","last_peered":"2024-09-15T12:28:55.143630+0000","last_clean":"2024-09-15T12:28:55.143630+0000","last_became_active":"2024-09-15T12:28:50.845175+0000","last_became_peered":"2024-09-15T12:28:50.845175+0000","last_unstale":"2024-09-15T12:28:55.143630+0000","last_undegraded":"2024-09-15T12:28:55.143630+0000","last_fullsized":"2024-09-15T12:28:55.143630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'512","reported_seq":576,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.205779+0000","last_change":"2024-09-15T12:28:50.813217+0000","last_active":"2024-09-15T12:28:55.205779+0000","last_peered":"2024-09-15T12:28:55.205779+0000","last_clean":"2024-09-15T12:28:55.205779+0000","last_became_active":"2024-09-15T12:28:50.802664+0000","last_became_peered":"2024-09-15T12:28:50.802664+0000","last_unstale":"2024-09-15T12:28:55.205779+0000","last_undegraded":"2024-09-15T12:28:55.205779+0000","last_fullsized":"2024-09-15T12:28:55.205779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":408,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'554","reported_seq":618,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.266489+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:28:55.266489+0000","last_peered":"2024-09-15T12:28:55.266489+0000","last_clean":"2024-09-15T12:28:55.266489+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:28:55.266489+0000","last_undegraded":"2024-09-15T12:28:55.266489+0000","last_fullsized":"2024-09-15T12:28:55.266489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"76'564","reported_seq":637,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.248741+0000","last_change":"2024-09-15T12:28:51.088903+0000","last_active":"2024-09-15T12:28:55.248741+0000","last_peered":"2024-09-15T12:28:55.248741+0000","last_clean":"2024-09-15T12:28:55.248741+0000","last_became_active":"2024-09-15T12:28:50.841053+0000","last_became_peered":"2024-09-15T12:28:50.841053+0000","last_unstale":"2024-09-15T12:28:55.248741+0000","last_undegraded":"2024-09-15T12:28:55.248741+0000","last_fullsized":"2024-09-15T12:28:55.248741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":564,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'557","reported_seq":629,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.283476+0000","last_change":"2024-09-15T12:28:50.793908+0000","last_active":"2024-09-15T12:28:55.283476+0000","last_peered":"2024-09-15T12:28:55.283476+0000","last_clean":"2024-09-15T12:28:55.283476+0000","last_became_active":"2024-09-15T12:28:50.786871+0000","last_became_peered":"2024-09-15T12:28:50.786871+0000","last_unstale":"2024-09-15T12:28:55.283476+0000","last_undegraded":"2024-09-15T12:28:55.283476+0000","last_fullsized":"2024-09-15T12:28:55.283476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":557,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'540","reported_seq":612,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.233274+0000","last_change":"2024-09-15T12:28:50.828793+0000","last_active":"2024-09-15T12:28:55.233274+0000","last_peered":"2024-09-15T12:28:55.233274+0000","last_clean":"2024-09-15T12:28:55.233274+0000","last_became_active":"2024-09-15T12:28:50.815603+0000","last_became_peered":"2024-09-15T12:28:50.815603+0000","last_unstale":"2024-09-15T12:28:55.233274+0000","last_undegraded":"2024-09-15T12:28:55.233274+0000","last_fullsized":"2024-09-15T12:28:55.233274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":410,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":540,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753035+0000","last_change":"2024-09-15T12:28:53.759286+0000","last_active":"2024-09-15T12:28:53.753035+0000","last_peered":"2024-09-15T12:28:53.753035+0000","last_clean":"2024-09-15T12:28:53.753035+0000","last_became_active":"2024-09-15T12:28:50.839450+0000","last_became_peered":"2024-09-15T12:28:50.839450+0000","last_unstale":"2024-09-15T12:28:53.753035+0000","last_undegraded":"2024-09-15T12:28:53.753035+0000","last_fullsized":"2024-09-15T12:28:53.753035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753811+0000","last_change":"2024-09-15T12:28:53.763391+0000","last_active":"2024-09-15T12:28:53.753811+0000","last_peered":"2024-09-15T12:28:53.753811+0000","last_clean":"2024-09-15T12:28:53.753811+0000","last_became_active":"2024-09-15T12:28:50.821740+0000","last_became_peered":"2024-09-15T12:28:50.821740+0000","last_unstale":"2024-09-15T12:28:53.753811+0000","last_undegraded":"2024-09-15T12:28:53.753811+0000","last_fullsized":"2024-09-15T12:28:53.753811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":397869080,"num_objects":6072,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73961472,"data_stored":1183252528,"data_compressed":56186291,"data_compressed_allocated":73953280,"data_compressed_original":1183252480,"omap_allocated":0,"internal_metadata":0},"log_size":9257,"ondisk_log_size":9257,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266806,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66312,"kb_used_data":25720,"kb_used_omap":0,"kb_used_meta":40576,"kb_avail":94305528,"statfs":{"total":96636764160,"available":96568860672,"internally_reserved":0,"allocated":26337280,"data_stored":410963602,"data_compressed":19504633,"data_compressed_allocated":25673728,"data_compressed_original":410697728,"omap_allocated":0,"internal_metadata":41549824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24735744,"data_stored":395706392,"data_compressed":18790157,"data_compressed_allocated":24731648,"data_compressed_original":395706368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:21.721 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:21.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:21.876+0000 7f1704911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:21.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:21.888+0000 7f1704911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:22.115 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:22.234 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:22.235 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:22.254 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-15T12:29:22.100014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398459467,"num_objects":6075,"num_object_clones":0,"num_object_copies":18225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":9335,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9312,"ondisk_log_size":9312,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":273488,"kb_used_data":75984,"kb_used_omap":0,"kb_used_meta":197440,"kb_avail":377213872,"statfs":{"total":386547056640,"available":386267004928,"internally_reserved":0,"allocated":77807616,"data_stored":1203779573,"data_compressed":57119591,"data_compressed_allocated":75186176,"data_compressed_original":1202733056,"omap_allocated":0,"internal_metadata":202178560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11272192,"num_objects":-172,"num_object_clones":0,"num_object_copies":-516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000674"},"pg_stats":[{"pgid":"5.f","version":"76'572","reported_seq":645,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.143630+0000","last_change":"2024-09-15T12:28:51.086796+0000","last_active":"2024-09-15T12:28:55.143630+0000","last_peered":"2024-09-15T12:28:55.143630+0000","last_clean":"2024-09-15T12:28:55.143630+0000","last_became_active":"2024-09-15T12:28:50.845175+0000","last_became_peered":"2024-09-15T12:28:50.845175+0000","last_unstale":"2024-09-15T12:28:55.143630+0000","last_undegraded":"2024-09-15T12:28:55.143630+0000","last_fullsized":"2024-09-15T12:28:55.143630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'512","reported_seq":576,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.205779+0000","last_change":"2024-09-15T12:28:50.813217+0000","last_active":"2024-09-15T12:28:55.205779+0000","last_peered":"2024-09-15T12:28:55.205779+0000","last_clean":"2024-09-15T12:28:55.205779+0000","last_became_active":"2024-09-15T12:28:50.802664+0000","last_became_peered":"2024-09-15T12:28:50.802664+0000","last_unstale":"2024-09-15T12:28:55.205779+0000","last_undegraded":"2024-09-15T12:28:55.205779+0000","last_fullsized":"2024-09-15T12:28:55.205779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":408,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'554","reported_seq":618,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.266489+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:28:55.266489+0000","last_peered":"2024-09-15T12:28:55.266489+0000","last_clean":"2024-09-15T12:28:55.266489+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:28:55.266489+0000","last_undegraded":"2024-09-15T12:28:55.266489+0000","last_fullsized":"2024-09-15T12:28:55.266489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"76'564","reported_seq":637,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.248741+0000","last_change":"2024-09-15T12:28:51.088903+0000","last_active":"2024-09-15T12:28:55.248741+0000","last_peered":"2024-09-15T12:28:55.248741+0000","last_clean":"2024-09-15T12:28:55.248741+0000","last_became_active":"2024-09-15T12:28:50.841053+0000","last_became_peered":"2024-09-15T12:28:50.841053+0000","last_unstale":"2024-09-15T12:28:55.248741+0000","last_undegraded":"2024-09-15T12:28:55.248741+0000","last_fullsized":"2024-09-15T12:28:55.248741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":564,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'557","reported_seq":629,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.283476+0000","last_change":"2024-09-15T12:28:50.793908+0000","last_active":"2024-09-15T12:28:55.283476+0000","last_peered":"2024-09-15T12:28:55.283476+0000","last_clean":"2024-09-15T12:28:55.283476+0000","last_became_active":"2024-09-15T12:28:50.786871+0000","last_became_peered":"2024-09-15T12:28:50.786871+0000","last_unstale":"2024-09-15T12:28:55.283476+0000","last_undegraded":"2024-09-15T12:28:55.283476+0000","last_fullsized":"2024-09-15T12:28:55.283476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":557,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'540","reported_seq":612,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.233274+0000","last_change":"2024-09-15T12:28:50.828793+0000","last_active":"2024-09-15T12:28:55.233274+0000","last_peered":"2024-09-15T12:28:55.233274+0000","last_clean":"2024-09-15T12:28:55.233274+0000","last_became_active":"2024-09-15T12:28:50.815603+0000","last_became_peered":"2024-09-15T12:28:50.815603+0000","last_unstale":"2024-09-15T12:28:55.233274+0000","last_undegraded":"2024-09-15T12:28:55.233274+0000","last_fullsized":"2024-09-15T12:28:55.233274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":410,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":540,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753035+0000","last_change":"2024-09-15T12:28:53.759286+0000","last_active":"2024-09-15T12:28:53.753035+0000","last_peered":"2024-09-15T12:28:53.753035+0000","last_clean":"2024-09-15T12:28:53.753035+0000","last_became_active":"2024-09-15T12:28:50.839450+0000","last_became_peered":"2024-09-15T12:28:50.839450+0000","last_unstale":"2024-09-15T12:28:53.753035+0000","last_undegraded":"2024-09-15T12:28:53.753035+0000","last_fullsized":"2024-09-15T12:28:53.753035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753811+0000","last_change":"2024-09-15T12:28:53.763391+0000","last_active":"2024-09-15T12:28:53.753811+0000","last_peered":"2024-09-15T12:28:53.753811+0000","last_clean":"2024-09-15T12:28:53.753811+0000","last_became_active":"2024-09-15T12:28:50.821740+0000","last_became_peered":"2024-09-15T12:28:50.821740+0000","last_unstale":"2024-09-15T12:28:53.753811+0000","last_undegraded":"2024-09-15T12:28:53.753811+0000","last_fullsized":"2024-09-15T12:28:53.753811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":397869080,"num_objects":6072,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73961472,"data_stored":1183252528,"data_compressed":56186291,"data_compressed_allocated":73953280,"data_compressed_original":1183252480,"omap_allocated":0,"internal_metadata":0},"log_size":9257,"ondisk_log_size":9257,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266806,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66312,"kb_used_data":25720,"kb_used_omap":0,"kb_used_meta":40576,"kb_avail":94305528,"statfs":{"total":96636764160,"available":96568860672,"internally_reserved":0,"allocated":26337280,"data_stored":410963602,"data_compressed":19504633,"data_compressed_allocated":25673728,"data_compressed_original":410697728,"omap_allocated":0,"internal_metadata":41549824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24735744,"data_stored":395706392,"data_compressed":18790157,"data_compressed_allocated":24731648,"data_compressed_original":395706368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:22.255 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:22.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:22.268+0000 7fa27cf11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:22.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:22.275+0000 7fa27cf11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:22.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:22.396+0000 7fe42885a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:22.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:22.407+0000 7fe42885a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:22.860 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":83,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":6075,"data_bytes":398459467,"bytes_used":280051712,"bytes_avail":386267004928,"bytes_total":386547056640,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:22.881 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:22.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:23.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:23.043+0000 7f8c976d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:23.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:23.056+0000 7f8c976d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:23.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:23.425 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:23.448 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-15T12:29:22.100014+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":398459467,"num_objects":6075,"num_object_clones":0,"num_object_copies":18225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":9335,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9312,"ondisk_log_size":9312,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":273488,"kb_used_data":75984,"kb_used_omap":0,"kb_used_meta":197440,"kb_avail":377213872,"statfs":{"total":386547056640,"available":386267004928,"internally_reserved":0,"allocated":77807616,"data_stored":1203779573,"data_compressed":57119591,"data_compressed_allocated":75186176,"data_compressed_original":1202733056,"omap_allocated":0,"internal_metadata":202178560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11272192,"num_objects":-172,"num_object_clones":0,"num_object_copies":-516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":172,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000674"},"pg_stats":[{"pgid":"5.f","version":"76'572","reported_seq":645,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.143630+0000","last_change":"2024-09-15T12:28:51.086796+0000","last_active":"2024-09-15T12:28:55.143630+0000","last_peered":"2024-09-15T12:28:55.143630+0000","last_clean":"2024-09-15T12:28:55.143630+0000","last_became_active":"2024-09-15T12:28:50.845175+0000","last_became_peered":"2024-09-15T12:28:50.845175+0000","last_unstale":"2024-09-15T12:28:55.143630+0000","last_undegraded":"2024-09-15T12:28:55.143630+0000","last_fullsized":"2024-09-15T12:28:55.143630+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":572,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"76'512","reported_seq":576,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.205779+0000","last_change":"2024-09-15T12:28:50.813217+0000","last_active":"2024-09-15T12:28:55.205779+0000","last_peered":"2024-09-15T12:28:55.205779+0000","last_clean":"2024-09-15T12:28:55.205779+0000","last_became_active":"2024-09-15T12:28:50.802664+0000","last_became_peered":"2024-09-15T12:28:50.802664+0000","last_unstale":"2024-09-15T12:28:55.205779+0000","last_undegraded":"2024-09-15T12:28:55.205779+0000","last_fullsized":"2024-09-15T12:28:55.205779+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26738688,"num_objects":408,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":512,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"76'554","reported_seq":618,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.266489+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:28:55.266489+0000","last_peered":"2024-09-15T12:28:55.266489+0000","last_clean":"2024-09-15T12:28:55.266489+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:28:55.266489+0000","last_undegraded":"2024-09-15T12:28:55.266489+0000","last_fullsized":"2024-09-15T12:28:55.266489+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":434,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":554,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"76'564","reported_seq":637,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.248741+0000","last_change":"2024-09-15T12:28:51.088903+0000","last_active":"2024-09-15T12:28:55.248741+0000","last_peered":"2024-09-15T12:28:55.248741+0000","last_clean":"2024-09-15T12:28:55.248741+0000","last_became_active":"2024-09-15T12:28:50.841053+0000","last_became_peered":"2024-09-15T12:28:50.841053+0000","last_unstale":"2024-09-15T12:28:55.248741+0000","last_undegraded":"2024-09-15T12:28:55.248741+0000","last_fullsized":"2024-09-15T12:28:55.248741+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28835840,"num_objects":440,"num_object_clones":0,"num_object_copies":1320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":564,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"76'557","reported_seq":629,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.283476+0000","last_change":"2024-09-15T12:28:50.793908+0000","last_active":"2024-09-15T12:28:55.283476+0000","last_peered":"2024-09-15T12:28:55.283476+0000","last_clean":"2024-09-15T12:28:55.283476+0000","last_became_active":"2024-09-15T12:28:50.786871+0000","last_became_peered":"2024-09-15T12:28:50.786871+0000","last_unstale":"2024-09-15T12:28:55.283476+0000","last_undegraded":"2024-09-15T12:28:55.283476+0000","last_fullsized":"2024-09-15T12:28:55.283476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":557,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"76'540","reported_seq":612,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:55.233274+0000","last_change":"2024-09-15T12:28:50.828793+0000","last_active":"2024-09-15T12:28:55.233274+0000","last_peered":"2024-09-15T12:28:55.233274+0000","last_clean":"2024-09-15T12:28:55.233274+0000","last_became_active":"2024-09-15T12:28:50.815603+0000","last_became_peered":"2024-09-15T12:28:50.815603+0000","last_unstale":"2024-09-15T12:28:55.233274+0000","last_undegraded":"2024-09-15T12:28:55.233274+0000","last_fullsized":"2024-09-15T12:28:55.233274+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":410,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":540,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753035+0000","last_change":"2024-09-15T12:28:53.759286+0000","last_active":"2024-09-15T12:28:53.753035+0000","last_peered":"2024-09-15T12:28:53.753035+0000","last_clean":"2024-09-15T12:28:53.753035+0000","last_became_active":"2024-09-15T12:28:50.839450+0000","last_became_peered":"2024-09-15T12:28:50.839450+0000","last_unstale":"2024-09-15T12:28:53.753035+0000","last_undegraded":"2024-09-15T12:28:53.753035+0000","last_fullsized":"2024-09-15T12:28:53.753035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.753811+0000","last_change":"2024-09-15T12:28:53.763391+0000","last_active":"2024-09-15T12:28:53.753811+0000","last_peered":"2024-09-15T12:28:53.753811+0000","last_clean":"2024-09-15T12:28:53.753811+0000","last_became_active":"2024-09-15T12:28:50.821740+0000","last_became_peered":"2024-09-15T12:28:50.821740+0000","last_unstale":"2024-09-15T12:28:53.753811+0000","last_undegraded":"2024-09-15T12:28:53.753811+0000","last_fullsized":"2024-09-15T12:28:53.753811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":397869080,"num_objects":6072,"num_object_clones":0,"num_object_copies":18216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73961472,"data_stored":1183252528,"data_compressed":56186291,"data_compressed_allocated":73953280,"data_compressed_original":1183252480,"omap_allocated":0,"internal_metadata":0},"log_size":9257,"ondisk_log_size":9257,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266806,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66312,"kb_used_data":25720,"kb_used_omap":0,"kb_used_meta":40576,"kb_avail":94305528,"statfs":{"total":96636764160,"available":96568860672,"internally_reserved":0,"allocated":26337280,"data_stored":410963602,"data_compressed":19504633,"data_compressed_allocated":25673728,"data_compressed_original":410697728,"omap_allocated":0,"internal_metadata":41549824},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24735744,"data_stored":395706392,"data_compressed":18790157,"data_compressed_allocated":24731648,"data_compressed_original":395706368,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:24.694 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:24.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:24.856+0000 7fd8a68d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:24.868 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:24.866+0000 7fd8a68d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:26.450 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:26.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:26.601+0000 7fdadeb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:26.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:26.613+0000 7fdadeb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:26.979 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:26.980 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:27.000 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-15T12:29:26.100776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-92405760,"num_objects":-1410,"num_object_clones":0,"num_object_copies":-4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001110"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:27.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:27.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.168+0000 7fa57be57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:27.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.182+0000 7fa57abf5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:27.297 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:27.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.459+0000 7f8429cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:27.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.466+0000 7f8429cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:27.689 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:27.689 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":4837,"data_bytes":317325899,"bytes_used":263815168,"bytes_avail":386283241472,"bytes_total":386547056640,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":234},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:27.709 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:27.709 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:27.867 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.856+0000 7fec8c6d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:27.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:27.871+0000 7fec8c6d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:28.185 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:28.185 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:28.209 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-15T12:29:28.101247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-92405760,"num_objects":-1410,"num_object_clones":0,"num_object_copies":-4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1410,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.907307"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:29.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:30.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:30.099+0000 7f52f0391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:30.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:30.108+0000 7f52f0391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:31.211 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:31.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:31.360+0000 7f6e82f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:31.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:31.372+0000 7f6e82f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:31.725 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:31.725 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:31.745 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T12:29:30.101662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-81133568,"num_objects":-1238,"num_object_clones":0,"num_object_copies":-3714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1238,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.901934"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:31.746 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:31.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:31.902+0000 7fb355f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:31.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:31.913+0000 7fb355f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:32.417 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:32.418 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":4837,"data_bytes":317325899,"bytes_used":263815168,"bytes_avail":386283241472,"bytes_total":386547056640,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":139},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:32.440 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:32.440 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:32.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:32.591+0000 7f1ced391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:32.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:32.606+0000 7f1ced391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:32.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:32.704+0000 7f3615591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:32.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:32.712+0000 7f3615591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:32.985 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:32.986 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-15T12:29:32.102095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-81133568,"num_objects":-1238,"num_object_clones":0,"num_object_copies":-3714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1238,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002080"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:34.914 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:35.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:35.068+0000 7f14e4951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:35.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:35.076+0000 7f14e4951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:36.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:36.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:36.166+0000 7fe42db91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:36.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:36.179+0000 7fe42db91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:36.525 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:36.525 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:36.546 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-15T12:29:36.102881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-81133568,"num_objects":-1238,"num_object_clones":0,"num_object_copies":-3714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1238,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002521"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:36.548 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:36.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:36.706+0000 7f41db591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:36.721+0000 7f41db591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:37.215 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:37.215 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":463,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":4837,"data_bytes":317325899,"bytes_used":263815168,"bytes_avail":386283241472,"bytes_total":386547056640,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":103},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:37.237 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:37.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:37.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:37.383+0000 7fab21f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:37.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:37.394+0000 7fab21f51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:37.499 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:37.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:37.639+0000 7fa2a8ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:37.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:37.647+0000 7fa2a8ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:37.750 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:37.750 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:37.773 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-15T12:29:36.102881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-81133568,"num_objects":-1238,"num_object_clones":0,"num_object_copies":-3714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1238,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002521"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:39.988 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:40.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:40.132+0000 7f231f0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:40.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:40.139+0000 7f231f0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:40.775 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:40.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:40.930+0000 7ff4a5191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:40.942+0000 7ff4a5191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:41.320 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:41.341 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-15T12:29:40.103623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002375"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:41.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:41.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:41.474+0000 7f6c81493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:41.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:41.481+0000 7f6c81493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:41.523 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Removed 7749 objects 2024-09-15T12:29:41.523 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Clean up completed and total clean up time :49.7021 2024-09-15T12:29:41.749 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:41.749 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":468,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":4837,"data_bytes":317325899,"bytes_used":263815168,"bytes_avail":386283241472,"bytes_total":386547056640},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:41.762 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:41.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:41.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:41.864+0000 7f9a38991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:41.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:41.871+0000 7f9a38991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:42.057 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:42.057 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:42.069 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-15T12:29:40.103623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317325899,"num_objects":4837,"num_object_clones":0,"num_object_copies":14511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":10573,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10550,"ondisk_log_size":10550,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":257632,"kb_used_data":64288,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377229728,"statfs":{"total":386547056640,"available":386283241472,"internally_reserved":0,"allocated":65830912,"data_stored":1011130859,"data_compressed":47970630,"data_compressed_allocated":63143936,"data_compressed_original":1010057216,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002375"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"76'609","reported_seq":673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.794020+0000","last_change":"2024-09-15T12:28:50.848570+0000","last_active":"2024-09-15T12:29:01.794020+0000","last_peered":"2024-09-15T12:29:01.794020+0000","last_clean":"2024-09-15T12:29:01.794020+0000","last_became_active":"2024-09-15T12:28:50.830267+0000","last_became_peered":"2024-09-15T12:28:50.830267+0000","last_unstale":"2024-09-15T12:29:01.794020+0000","last_undegraded":"2024-09-15T12:29:01.794020+0000","last_fullsized":"2024-09-15T12:29:01.794020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22085632,"num_objects":337,"num_object_clones":0,"num_object_copies":1011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":609,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"76'602","reported_seq":672,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.692173+0000","last_change":"2024-09-15T12:28:51.019722+0000","last_active":"2024-09-15T12:29:01.692173+0000","last_peered":"2024-09-15T12:29:01.692173+0000","last_clean":"2024-09-15T12:29:01.692173+0000","last_became_active":"2024-09-15T12:28:50.839276+0000","last_became_peered":"2024-09-15T12:28:50.839276+0000","last_unstale":"2024-09-15T12:29:01.692173+0000","last_undegraded":"2024-09-15T12:29:01.692173+0000","last_fullsized":"2024-09-15T12:29:01.692173+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119576,"num_objects":308,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":602,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"76'626","reported_seq":691,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.679317+0000","last_change":"2024-09-15T12:28:50.828421+0000","last_active":"2024-09-15T12:29:01.679317+0000","last_peered":"2024-09-15T12:29:01.679317+0000","last_clean":"2024-09-15T12:29:01.679317+0000","last_became_active":"2024-09-15T12:28:50.809880+0000","last_became_peered":"2024-09-15T12:28:50.809880+0000","last_unstale":"2024-09-15T12:29:01.679317+0000","last_undegraded":"2024-09-15T12:29:01.679317+0000","last_fullsized":"2024-09-15T12:29:01.679317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":626,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"76'684","reported_seq":759,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.644463+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:01.644463+0000","last_peered":"2024-09-15T12:29:01.644463+0000","last_clean":"2024-09-15T12:29:01.644463+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:01.644463+0000","last_undegraded":"2024-09-15T12:29:01.644463+0000","last_fullsized":"2024-09-15T12:29:01.644463+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":358,"num_object_clones":0,"num_object_copies":1074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":684,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"76'673","reported_seq":737,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.523895+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:01.523895+0000","last_peered":"2024-09-15T12:29:01.523895+0000","last_clean":"2024-09-15T12:29:01.523895+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:01.523895+0000","last_undegraded":"2024-09-15T12:29:01.523895+0000","last_fullsized":"2024-09-15T12:29:01.523895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":341,"num_object_clones":0,"num_object_copies":1023,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":673,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.750163+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:28:53.750163+0000","last_peered":"2024-09-15T12:28:53.750163+0000","last_clean":"2024-09-15T12:28:53.750163+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:28:53.750163+0000","last_undegraded":"2024-09-15T12:28:53.750163+0000","last_fullsized":"2024-09-15T12:28:53.750163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":196,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:28:53.739036+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:28:53.739036+0000","last_peered":"2024-09-15T12:28:53.739036+0000","last_clean":"2024-09-15T12:28:53.739036+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:28:53.739036+0000","last_undegraded":"2024-09-15T12:28:53.739036+0000","last_fullsized":"2024-09-15T12:28:53.739036+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"76'617","reported_seq":696,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.759368+0000","last_change":"2024-09-15T12:28:50.980642+0000","last_active":"2024-09-15T12:29:01.759368+0000","last_peered":"2024-09-15T12:29:01.759368+0000","last_clean":"2024-09-15T12:29:01.759368+0000","last_became_active":"2024-09-15T12:28:50.797680+0000","last_became_peered":"2024-09-15T12:28:50.797680+0000","last_unstale":"2024-09-15T12:29:01.759368+0000","last_undegraded":"2024-09-15T12:29:01.759368+0000","last_fullsized":"2024-09-15T12:29:01.759368+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":617,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"76'583","reported_seq":652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T12:29:01.778206+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:01.778206+0000","last_peered":"2024-09-15T12:29:01.778206+0000","last_clean":"2024-09-15T12:29:01.778206+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:01.778206+0000","last_undegraded":"2024-09-15T12:29:01.778206+0000","last_fullsized":"2024-09-15T12:29:01.778206+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":291,"num_object_clones":0,"num_object_copies":873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":316735512,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10494,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62091264,"data_stored":993329200,"data_compressed":47167814,"data_compressed_allocated":62083072,"data_compressed_original":993329152,"omap_allocated":0,"internal_metadata":0},"log_size":10495,"ondisk_log_size":10495,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":75444,"kb_used_data":17764,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94296396,"statfs":{"total":96636764160,"available":96559509504,"internally_reserved":0,"allocated":18190336,"data_stored":280612498,"data_compressed":13315184,"data_compressed_allocated":17526784,"data_compressed_original":280346624,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17489920,"data_stored":279773208,"data_compressed":13285128,"data_compressed_allocated":17485824,"data_compressed_original":279773184,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:42.363 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:42.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:42.465+0000 7ff4ffea3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:42.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:42.471+0000 7ff4ffea3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:44.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:44.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:44.688+0000 7f95330bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:44.696 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:44.694+0000 7f95330bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:45.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.175+0000 7fd11b191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.182+0000 7fd11b191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.367 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:45.367 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:45.379 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-15T12:29:44.104533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170984011,"num_objects":2604,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":12806,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12783,"ondisk_log_size":12783,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":241256,"kb_used_data":47908,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377246104,"statfs":{"total":386547056640,"available":386300010496,"internally_reserved":0,"allocated":49057792,"data_stored":741739489,"data_compressed":35177198,"data_compressed_allocated":46305280,"data_compressed_original":740638720,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-146341888,"num_objects":-2233,"num_object_clones":0,"num_object_copies":-6699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002438"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'938","reported_seq":1036,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.859342+0000","last_change":"2024-09-15T12:29:17.538019+0000","last_active":"2024-09-15T12:29:40.859342+0000","last_peered":"2024-09-15T12:29:40.859342+0000","last_clean":"2024-09-15T12:29:40.859342+0000","last_became_active":"2024-09-15T12:29:17.382739+0000","last_became_peered":"2024-09-15T12:29:17.382739+0000","last_unstale":"2024-09-15T12:29:40.859342+0000","last_undegraded":"2024-09-15T12:29:40.859342+0000","last_fullsized":"2024-09-15T12:29:40.859342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"84'903","reported_seq":1004,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.825801+0000","last_change":"2024-09-15T12:29:17.604371+0000","last_active":"2024-09-15T12:29:40.825801+0000","last_peered":"2024-09-15T12:29:40.825801+0000","last_clean":"2024-09-15T12:29:40.825801+0000","last_became_active":"2024-09-15T12:29:17.398880+0000","last_became_peered":"2024-09-15T12:29:17.398880+0000","last_unstale":"2024-09-15T12:29:40.825801+0000","last_undegraded":"2024-09-15T12:29:40.825801+0000","last_fullsized":"2024-09-15T12:29:40.825801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393240,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":903,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"84'945","reported_seq":1035,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.912673+0000","last_change":"2024-09-15T12:29:17.361673+0000","last_active":"2024-09-15T12:29:40.912673+0000","last_peered":"2024-09-15T12:29:40.912673+0000","last_clean":"2024-09-15T12:29:40.912673+0000","last_became_active":"2024-09-15T12:29:17.344129+0000","last_became_peered":"2024-09-15T12:29:17.344129+0000","last_unstale":"2024-09-15T12:29:40.912673+0000","last_undegraded":"2024-09-15T12:29:40.912673+0000","last_fullsized":"2024-09-15T12:29:40.912673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":945,"log_dups_size":0,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"84'1039","reported_seq":1138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.712895+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:40.712895+0000","last_peered":"2024-09-15T12:29:40.712895+0000","last_clean":"2024-09-15T12:29:40.712895+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:40.712895+0000","last_undegraded":"2024-09-15T12:29:40.712895+0000","last_fullsized":"2024-09-15T12:29:40.712895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":1039,"log_dups_size":0,"ondisk_log_size":1039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1039,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"84'1010","reported_seq":1098,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.945039+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:40.945039+0000","last_peered":"2024-09-15T12:29:40.945039+0000","last_clean":"2024-09-15T12:29:40.945039+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:40.945039+0000","last_undegraded":"2024-09-15T12:29:40.945039+0000","last_fullsized":"2024-09-15T12:29:40.945039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1010,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"84'921","reported_seq":1031,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.935002+0000","last_change":"2024-09-15T12:29:17.466170+0000","last_active":"2024-09-15T12:29:40.935002+0000","last_peered":"2024-09-15T12:29:40.935002+0000","last_clean":"2024-09-15T12:29:40.935002+0000","last_became_active":"2024-09-15T12:29:17.332387+0000","last_became_peered":"2024-09-15T12:29:17.332387+0000","last_unstale":"2024-09-15T12:29:40.935002+0000","last_undegraded":"2024-09-15T12:29:40.935002+0000","last_fullsized":"2024-09-15T12:29:40.935002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'871","reported_seq":964,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.963136+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:40.963136+0000","last_peered":"2024-09-15T12:29:40.963136+0000","last_clean":"2024-09-15T12:29:40.963136+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:40.963136+0000","last_undegraded":"2024-09-15T12:29:40.963136+0000","last_fullsized":"2024-09-15T12:29:40.963136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":871,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":170393624,"num_objects":2601,"num_object_clones":0,"num_object_copies":7803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12727,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":44900352,"data_stored":718274608,"data_compressed":34106750,"data_compressed_allocated":44892160,"data_compressed_original":718274560,"omap_allocated":0,"internal_metadata":0},"log_size":12728,"ondisk_log_size":12728,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":4718616,"data_compressed":224064,"data_compressed_allocated":294912,"data_compressed_original":4718592,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:45.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:45.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.482+0000 7f7b24b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.489+0000 7f7b24b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.745 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:45.745 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":472,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":84,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"stale+active+clean","count":4}],"num_pgs":25,"num_pools":3,"num_objects":2604,"data_bytes":170984011,"bytes_used":247046144,"bytes_avail":386300010496,"bytes_total":386547056640,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":186},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:45.758 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:45.758 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:45.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.859+0000 7f127447d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:45.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:45.867+0000 7f127447d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:46.054 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:46.054 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:46.066 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-15T12:29:44.104533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170984011,"num_objects":2604,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":12806,"num_write_kb":491541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12783,"ondisk_log_size":12783,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":241256,"kb_used_data":47908,"kb_used_omap":0,"kb_used_meta":193280,"kb_avail":377246104,"statfs":{"total":386547056640,"available":386300010496,"internally_reserved":0,"allocated":49057792,"data_stored":741739489,"data_compressed":35177198,"data_compressed_allocated":46305280,"data_compressed_original":740638720,"omap_allocated":0,"internal_metadata":197918720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-146341888,"num_objects":-2233,"num_object_clones":0,"num_object_copies":-6699,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002438"},"pg_stats":[{"pgid":"5.f","version":"84'792","reported_seq":890,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.282124+0000","last_change":"2024-09-15T12:29:17.344929+0000","last_active":"2024-09-15T12:29:25.282124+0000","last_peered":"2024-09-15T12:29:25.282124+0000","last_clean":"2024-09-15T12:29:25.282124+0000","last_became_active":"2024-09-15T12:29:17.336317+0000","last_became_peered":"2024-09-15T12:29:17.336317+0000","last_unstale":"2024-09-15T12:29:25.282124+0000","last_undegraded":"2024-09-15T12:29:25.282124+0000","last_fullsized":"2024-09-15T12:29:25.282124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":792,"log_dups_size":0,"ondisk_log_size":792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15073280,"num_objects":230,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":792,"num_write_kb":32704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"84'938","reported_seq":1036,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.859342+0000","last_change":"2024-09-15T12:29:17.538019+0000","last_active":"2024-09-15T12:29:40.859342+0000","last_peered":"2024-09-15T12:29:40.859342+0000","last_clean":"2024-09-15T12:29:40.859342+0000","last_became_active":"2024-09-15T12:29:17.382739+0000","last_became_peered":"2024-09-15T12:29:17.382739+0000","last_unstale":"2024-09-15T12:29:40.859342+0000","last_undegraded":"2024-09-15T12:29:40.859342+0000","last_fullsized":"2024-09-15T12:29:40.859342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524288,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"84'903","reported_seq":1004,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.825801+0000","last_change":"2024-09-15T12:29:17.604371+0000","last_active":"2024-09-15T12:29:40.825801+0000","last_peered":"2024-09-15T12:29:40.825801+0000","last_clean":"2024-09-15T12:29:40.825801+0000","last_became_active":"2024-09-15T12:29:17.398880+0000","last_became_peered":"2024-09-15T12:29:17.398880+0000","last_unstale":"2024-09-15T12:29:40.825801+0000","last_undegraded":"2024-09-15T12:29:40.825801+0000","last_fullsized":"2024-09-15T12:29:40.825801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393240,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":903,"num_write_kb":29057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.c","version":"71'486","reported_seq":535,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.367873+0000","last_change":"2024-09-15T12:27:51.775007+0000","last_active":"2024-09-15T12:28:21.367873+0000","last_peered":"2024-09-15T12:28:21.367873+0000","last_clean":"2024-09-15T12:28:21.367873+0000","last_became_active":"2024-09-15T12:27:51.767138+0000","last_became_peered":"2024-09-15T12:27:51.767138+0000","last_unstale":"2024-09-15T12:28:48.695985+0000","last_undegraded":"2024-09-15T12:28:21.367873+0000","last_fullsized":"2024-09-15T12:28:21.367873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":486,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":486,"num_write_kb":31104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"84'945","reported_seq":1035,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.912673+0000","last_change":"2024-09-15T12:29:17.361673+0000","last_active":"2024-09-15T12:29:40.912673+0000","last_peered":"2024-09-15T12:29:40.912673+0000","last_clean":"2024-09-15T12:29:40.912673+0000","last_became_active":"2024-09-15T12:29:17.344129+0000","last_became_peered":"2024-09-15T12:29:17.344129+0000","last_unstale":"2024-09-15T12:29:40.912673+0000","last_undegraded":"2024-09-15T12:29:40.912673+0000","last_fullsized":"2024-09-15T12:29:40.912673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":945,"log_dups_size":0,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":945,"num_write_kb":30336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"84'712","reported_seq":810,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.037545+0000","last_change":"2024-09-15T12:29:17.627557+0000","last_active":"2024-09-15T12:29:25.037545+0000","last_peered":"2024-09-15T12:29:25.037545+0000","last_clean":"2024-09-15T12:29:25.037545+0000","last_became_active":"2024-09-15T12:29:17.337477+0000","last_became_peered":"2024-09-15T12:29:17.337477+0000","last_unstale":"2024-09-15T12:29:25.037545+0000","last_undegraded":"2024-09-15T12:29:25.037545+0000","last_fullsized":"2024-09-15T12:29:25.037545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":208,"num_object_clones":0,"num_object_copies":624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"82'633","reported_seq":713,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.472449+0000","last_change":"2024-09-15T12:27:51.913177+0000","last_active":"2024-09-15T12:29:20.472449+0000","last_peered":"2024-09-15T12:29:20.472449+0000","last_clean":"2024-09-15T12:29:20.472449+0000","last_became_active":"2024-09-15T12:27:51.872280+0000","last_became_peered":"2024-09-15T12:27:51.872280+0000","last_unstale":"2024-09-15T12:29:20.472449+0000","last_undegraded":"2024-09-15T12:29:20.472449+0000","last_fullsized":"2024-09-15T12:29:20.472449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":633,"log_dups_size":0,"ondisk_log_size":633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18874368,"num_objects":288,"num_object_clones":0,"num_object_copies":864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":632,"num_write_kb":29440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.8","version":"84'1039","reported_seq":1138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.712895+0000","last_change":"2024-09-15T12:27:52.123600+0000","last_active":"2024-09-15T12:29:40.712895+0000","last_peered":"2024-09-15T12:29:40.712895+0000","last_clean":"2024-09-15T12:29:40.712895+0000","last_became_active":"2024-09-15T12:27:51.857252+0000","last_became_peered":"2024-09-15T12:27:51.857252+0000","last_unstale":"2024-09-15T12:29:40.712895+0000","last_undegraded":"2024-09-15T12:29:40.712895+0000","last_fullsized":"2024-09-15T12:29:40.712895+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":1039,"log_dups_size":0,"ondisk_log_size":1039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1039,"num_write_kb":33344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"84'745","reported_seq":833,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.244833+0000","last_change":"2024-09-15T12:27:51.796731+0000","last_active":"2024-09-15T12:29:25.244833+0000","last_peered":"2024-09-15T12:29:25.244833+0000","last_clean":"2024-09-15T12:29:25.244833+0000","last_became_active":"2024-09-15T12:27:51.789855+0000","last_became_peered":"2024-09-15T12:27:51.789855+0000","last_unstale":"2024-09-15T12:29:25.244833+0000","last_undegraded":"2024-09-15T12:29:25.244833+0000","last_fullsized":"2024-09-15T12:29:25.244833+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925248,"num_objects":243,"num_object_clones":0,"num_object_copies":729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":745,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":210,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.711700+0000","last_change":"2024-09-15T12:26:57.264317+0000","last_active":"2024-09-15T12:28:18.711700+0000","last_peered":"2024-09-15T12:28:18.711700+0000","last_clean":"2024-09-15T12:28:18.711700+0000","last_became_active":"2024-09-15T12:26:57.258511+0000","last_became_peered":"2024-09-15T12:26:57.258511+0000","last_unstale":"2024-09-15T12:28:48.695987+0000","last_undegraded":"2024-09-15T12:28:18.711700+0000","last_fullsized":"2024-09-15T12:28:18.711700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"84'1010","reported_seq":1098,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.945039+0000","last_change":"2024-09-15T12:27:51.926950+0000","last_active":"2024-09-15T12:29:40.945039+0000","last_peered":"2024-09-15T12:29:40.945039+0000","last_clean":"2024-09-15T12:29:40.945039+0000","last_became_active":"2024-09-15T12:27:51.901122+0000","last_became_peered":"2024-09-15T12:27:51.901122+0000","last_unstale":"2024-09-15T12:29:40.945039+0000","last_undegraded":"2024-09-15T12:29:40.945039+0000","last_fullsized":"2024-09-15T12:29:40.945039+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1010,"num_write_kb":32448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"84'775","reported_seq":882,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.143296+0000","last_change":"2024-09-15T12:29:17.649728+0000","last_active":"2024-09-15T12:29:25.143296+0000","last_peered":"2024-09-15T12:29:25.143296+0000","last_clean":"2024-09-15T12:29:25.143296+0000","last_became_active":"2024-09-15T12:29:17.370210+0000","last_became_peered":"2024-09-15T12:29:17.370210+0000","last_unstale":"2024-09-15T12:29:25.143296+0000","last_undegraded":"2024-09-15T12:29:25.143296+0000","last_fullsized":"2024-09-15T12:29:25.143296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":229,"num_object_clones":0,"num_object_copies":687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":32128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"84'781","reported_seq":887,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.114490+0000","last_change":"2024-09-15T12:29:17.490725+0000","last_active":"2024-09-15T12:29:25.114490+0000","last_peered":"2024-09-15T12:29:25.114490+0000","last_clean":"2024-09-15T12:29:25.114490+0000","last_became_active":"2024-09-15T12:29:17.299742+0000","last_became_peered":"2024-09-15T12:29:17.299742+0000","last_unstale":"2024-09-15T12:29:25.114490+0000","last_undegraded":"2024-09-15T12:29:25.114490+0000","last_fullsized":"2024-09-15T12:29:25.114490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":221,"num_object_clones":0,"num_object_copies":663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":781,"num_write_kb":32064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"84'732","reported_seq":835,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:25.154984+0000","last_change":"2024-09-15T12:29:17.591930+0000","last_active":"2024-09-15T12:29:25.154984+0000","last_peered":"2024-09-15T12:29:25.154984+0000","last_clean":"2024-09-15T12:29:25.154984+0000","last_became_active":"2024-09-15T12:29:17.316312+0000","last_became_peered":"2024-09-15T12:29:17.316312+0000","last_unstale":"2024-09-15T12:29:25.154984+0000","last_undegraded":"2024-09-15T12:29:25.154984+0000","last_fullsized":"2024-09-15T12:29:25.154984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":732,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"84'921","reported_seq":1031,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.935002+0000","last_change":"2024-09-15T12:29:17.466170+0000","last_active":"2024-09-15T12:29:40.935002+0000","last_peered":"2024-09-15T12:29:40.935002+0000","last_clean":"2024-09-15T12:29:40.935002+0000","last_became_active":"2024-09-15T12:29:17.332387+0000","last_became_peered":"2024-09-15T12:29:17.332387+0000","last_unstale":"2024-09-15T12:29:40.935002+0000","last_undegraded":"2024-09-15T12:29:40.935002+0000","last_fullsized":"2024-09-15T12:29:40.935002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":921,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":147,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:18.720045+0000","last_change":"2024-09-15T12:26:58.169469+0000","last_active":"2024-09-15T12:28:18.720045+0000","last_peered":"2024-09-15T12:28:18.720045+0000","last_clean":"2024-09-15T12:28:18.720045+0000","last_became_active":"2024-09-15T12:26:57.290066+0000","last_became_peered":"2024-09-15T12:26:57.290066+0000","last_unstale":"2024-09-15T12:28:48.695989+0000","last_undegraded":"2024-09-15T12:28:18.720045+0000","last_fullsized":"2024-09-15T12:28:18.720045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"71'445","reported_seq":493,"reported_epoch":71,"state":"stale+active+clean","last_fresh":"2024-09-15T12:28:21.256790+0000","last_change":"2024-09-15T12:27:19.849251+0000","last_active":"2024-09-15T12:28:21.256790+0000","last_peered":"2024-09-15T12:28:21.256790+0000","last_clean":"2024-09-15T12:28:21.256790+0000","last_became_active":"2024-09-15T12:27:19.843364+0000","last_became_peered":"2024-09-15T12:27:19.843364+0000","last_unstale":"2024-09-15T12:28:48.695990+0000","last_undegraded":"2024-09-15T12:28:21.256790+0000","last_fullsized":"2024-09-15T12:28:21.256790+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29163520,"num_objects":445,"num_object_clones":0,"num_object_copies":1335,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":445,"num_write_kb":28480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.0","version":"84'871","reported_seq":964,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:40.963136+0000","last_change":"2024-09-15T12:27:52.129814+0000","last_active":"2024-09-15T12:29:40.963136+0000","last_peered":"2024-09-15T12:29:40.963136+0000","last_clean":"2024-09-15T12:29:40.963136+0000","last_became_active":"2024-09-15T12:27:51.888089+0000","last_became_peered":"2024-09-15T12:27:51.888089+0000","last_unstale":"2024-09-15T12:29:40.963136+0000","last_undegraded":"2024-09-15T12:29:40.963136+0000","last_fullsized":"2024-09-15T12:29:40.963136+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:27:18.686229+0000","last_clean_scrub_stamp":"2024-09-15T12:27:18.686229+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":871,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":16,"stat_sum":{"num_bytes":170393624,"num_objects":2601,"num_object_clones":0,"num_object_copies":7803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12727,"num_write_kb":490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":44900352,"data_stored":718274608,"data_compressed":34106750,"data_compressed_allocated":44892160,"data_compressed_original":718274560,"omap_allocated":0,"internal_metadata":0},"log_size":12728,"ondisk_log_size":12728,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":35,"seq":150323855600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77972,"kb_used_data":19844,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94293868,"statfs":{"total":96636764160,"available":96556920832,"internally_reserved":0,"allocated":20320256,"data_stored":315460308,"data_compressed":14969998,"data_compressed_allocated":19705856,"data_compressed_original":315211776,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299008,"data_stored":4718616,"data_compressed":224064,"data_compressed_allocated":294912,"data_compressed_original":4718592,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11841536,"data_stored":189464576,"data_compressed":8996792,"data_compressed_allocated":11841536,"data_compressed_original":189464576,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":205783064,"data_compressed":9771680,"data_compressed_allocated":12861440,"data_compressed_original":205783040,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19894272,"data_stored":318308352,"data_compressed":15114214,"data_compressed_allocated":19894272,"data_compressed_original":318308352,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:46.800 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:46.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:46.908+0000 7f377a311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:46.913+0000 7f377a311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:47.156 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-15T12:29:47.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-15T12:29:47.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:47.260+0000 7f5442191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:47.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:47.271+0000 7f5442191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_2' removed 2024-09-15T12:29:47.726 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-15T12:29:47.727 INFO:tasks.radosbench:Beginning radosbench... 2024-09-15T12:29:47.727 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-15T12:29:47.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_3 16 2024-09-15T12:29:47.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:47.863+0000 7fa3674d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:47.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:47.877+0000 7fa3674d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:48.700 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_3' created 2024-09-15T12:29:48.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-15T12:29:48.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:48.846+0000 7fda5f991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:48.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:48.860+0000 7fda5f991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:49.068 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:49.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.120+0000 7fc6ec391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.126+0000 7fc6ec391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.184+0000 7f2476591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.191+0000 7f2476591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.386 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:49.399 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-15T12:29:48.701547+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":79,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":197388,"kb_used_data":28808,"kb_used_omap":0,"kb_used_meta":168512,"kb_avail":377289972,"statfs":{"total":386547056640,"available":386344931328,"internally_reserved":0,"allocated":29499392,"data_stored":427145578,"data_compressed":20237256,"data_compressed_allocated":26640384,"data_compressed_original":426000384,"omap_allocated":0,"internal_metadata":172556288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-316735512,"num_objects":-4834,"num_object_clones":0,"num_object_copies":-14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-10494,"num_write_kb":-490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-20,"num_bytes_recovered":-1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.597923"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:49.400 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:49.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.503+0000 7ff23f111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:49.509+0000 7ff23f111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:49.723 INFO:teuthology.orchestra.run.smithi103.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-15T12:29:49.816 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:49.816 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":87,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9}],"num_pgs":9,"num_pools":2,"num_objects":3,"data_bytes":590387,"bytes_used":202125312,"bytes_avail":386344931328,"bytes_total":386547056640},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:49.838 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:49.838 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:50.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:50.005+0000 7fc1f9991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:50.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:50.014+0000 7fc1f9991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:50.200 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:50.201 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:50.213 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-15T12:29:50.105797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":79,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":197388,"kb_used_data":28808,"kb_used_omap":0,"kb_used_meta":168512,"kb_avail":377289972,"statfs":{"total":386547056640,"available":386344931328,"internally_reserved":0,"allocated":29499392,"data_stored":427145578,"data_compressed":20237256,"data_compressed_allocated":26640384,"data_compressed_original":426000384,"omap_allocated":0,"internal_metadata":172556288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-170393624,"num_objects":-2601,"num_object_clones":0,"num_object_copies":-7803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-12727,"num_write_kb":-490433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-20,"num_bytes_recovered":-1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001263"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.240974+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:20.240974+0000","last_peered":"2024-09-15T12:29:20.240974+0000","last_clean":"2024-09-15T12:29:20.240974+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:20.240974+0000","last_undegraded":"2024-09-15T12:29:20.240974+0000","last_fullsized":"2024-09-15T12:29:20.240974+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.231696+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:20.231696+0000","last_peered":"2024-09-15T12:29:20.231696+0000","last_clean":"2024-09-15T12:29:20.231696+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:20.231696+0000","last_undegraded":"2024-09-15T12:29:20.231696+0000","last_fullsized":"2024-09-15T12:29:20.231696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":176,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-15T12:29:20.235120+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:20.235120+0000","last_peered":"2024-09-15T12:29:20.235120+0000","last_clean":"2024-09-15T12:29:20.235120+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:20.235120+0000","last_undegraded":"2024-09-15T12:29:20.235120+0000","last_fullsized":"2024-09-15T12:29:20.235120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005145,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":53760,"kb_used_data":12656,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94318080,"statfs":{"total":96636764160,"available":96581713920,"internally_reserved":0,"allocated":12959744,"data_stored":196743165,"data_compressed":9329776,"data_compressed_allocated":12279808,"data_compressed_original":196476928,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:50.746 DEBUG:teuthology.orchestra.run.smithi103:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-15T12:29:50.749 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-15T12:29:50.749 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-15T12:29:50.810 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:hints = 1 2024-09-15T12:29:50.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-15T12:29:50.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object prefix: benchmark_data_smithi103_40346 2024-09-15T12:29:50.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:29:50.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 0 0 0 0 0 0 - 0 2024-09-15T12:29:51.233 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:51.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:51.395+0000 7fdcac191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:51.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:51.404+0000 7fdcac191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:51.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1 16 242 226 14.1242 14.125 0.071904 0.0681596 2024-09-15T12:29:52.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2 16 470 454 14.1865 14.25 0.0424968 0.0690069 2024-09-15T12:29:53.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:53.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:53.392+0000 7ff175ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:53.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:53.404+0000 7ff175ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:53.792 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:53.793 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:53.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:53.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3 16 667 651 13.5611 12.3125 0.0271268 0.0721823 2024-09-15T12:29:53.813 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-15T12:29:52.106118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":79,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":181288,"kb_used_data":16868,"kb_used_omap":0,"kb_used_meta":164352,"kb_avail":377306072,"statfs":{"total":386547056640,"available":386361417728,"internally_reserved":0,"allocated":17272832,"data_stored":230688963,"data_compressed":10907480,"data_compressed_allocated":14360576,"data_compressed_original":229523456,"omap_allocated":0,"internal_metadata":168296448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.401306"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:53.815 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:53.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:53.965+0000 7f17b9548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:53.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:53.976+0000 7f17b9548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:54.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:54.017+0000 7fdbe92d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:54.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:54.032+0000 7fdbe92d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:54.559 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:54.559 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":480,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":88,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"unknown","count":12}],"num_pgs":25,"num_pools":3,"num_objects":3,"data_bytes":590387,"bytes_used":185638912,"bytes_avail":386361417728,"bytes_total":386547056640,"unknown_pgs_ratio":0.47999998927116394},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:54.583 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:54.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:54.736 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:54.726+0000 7fa984b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:54.742+0000 7fa984b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:54.811 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 4 16 861 845 13.2018 12.125 0.0481406 0.0751505 2024-09-15T12:29:55.057 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:55.057 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:55.078 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-15T12:29:54.106540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":79,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":181288,"kb_used_data":16868,"kb_used_omap":0,"kb_used_meta":164352,"kb_avail":377306072,"statfs":{"total":386547056640,"available":386361417728,"internally_reserved":0,"allocated":17272832,"data_stored":230688963,"data_compressed":10907480,"data_compressed_allocated":14360576,"data_compressed_original":229523456,"omap_allocated":0,"internal_metadata":168296448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.731090+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:23.731090+0000","last_peered":"2024-09-15T12:29:23.731090+0000","last_clean":"2024-09-15T12:29:23.731090+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:23.731090+0000","last_undegraded":"2024-09-15T12:29:23.731090+0000","last_fullsized":"2024-09-15T12:29:23.731090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":212,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.718587+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:23.718587+0000","last_peered":"2024-09-15T12:29:23.718587+0000","last_clean":"2024-09-15T12:29:23.718587+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:23.718587+0000","last_undegraded":"2024-09-15T12:29:23.718587+0000","last_fullsized":"2024-09-15T12:29:23.718587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50456,"kb_used_data":14024,"kb_used_omap":0,"kb_used_meta":36416,"kb_avail":94321384,"statfs":{"total":96636764160,"available":96585097216,"internally_reserved":0,"allocated":14360576,"data_stored":218314888,"data_compressed":10355672,"data_compressed_allocated":13631488,"data_compressed_original":218021888,"omap_allocated":0,"internal_metadata":37289984},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:55.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 5 16 1089 1073 13.4112 14.25 0.0725563 0.0737303 2024-09-15T12:29:56.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:29:56.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:56.483+0000 7f9cc1951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:56.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:56.494+0000 7f9cc1951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:56.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 6 16 1315 1299 13.5299 14.125 0.0659725 0.0735402 2024-09-15T12:29:57.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 7 16 1546 1530 13.6594 14.4375 0.0891988 0.0727592 2024-09-15T12:29:58.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.232+0000 7f27f4d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:58.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.250+0000 7f27f4d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:58.555 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:58.556 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:58.576 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-15T12:29:58.107428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.384701"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:58.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:29:58.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:29:58.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.736+0000 7fd369151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:58.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.751+0000 7fd369151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:58.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 8 16 1761 1745 13.6315 13.4375 0.120674 0.0729375 2024-09-15T12:29:58.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.892+0000 7fed58991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:58.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:58.907+0000 7fed58991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:59.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:59.173 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":485,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":88,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"unknown","count":7}],"num_pgs":25,"num_pools":3,"num_objects":321,"data_bytes":21430835,"bytes_used":192856064,"bytes_avail":386354200576,"bytes_total":386547056640,"unknown_pgs_ratio":0.2800000011920929,"write_bytes_sec":2485532,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:29:59.195 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:29:59.196 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:29:59.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:59.393+0000 7f310e488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:59.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:29:59.406+0000 7f310e488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:29:59.740 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:29:59.741 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:29:59.764 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-15T12:29:58.107428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.384701"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:29:59.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 9 16 1977 1961 13.6168 13.5 0.0728827 0.0731787 2024-09-15T12:30:00.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 10 16 2209 2193 13.705 14.5 0.0645352 0.0726747 2024-09-15T12:30:01.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:01.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:01.449+0000 7f6198711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:01.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:01.460+0000 7f6198711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:01.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 11 16 2427 2411 13.6976 13.625 0.0512849 0.0727655 2024-09-15T12:30:02.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:02.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 12 16 2647 2631 13.7019 13.75 0.0807561 0.0727095 2024-09-15T12:30:02.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:02.919+0000 7f56aa991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:02.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:02.933+0000 7f56aa991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:03.286 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:03.287 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:03.308 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-15T12:30:02.108280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.382436"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:03.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:30:03.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:03.472+0000 7ff6ad25c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:03.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:03.485+0000 7ff6ad25c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:03.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:03.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 13 16 2853 2837 13.6382 12.875 0.109728 0.073084 2024-09-15T12:30:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:03.898 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":88,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"unknown","count":7}],"num_pgs":25,"num_pools":3,"num_objects":321,"data_bytes":21430835,"bytes_used":192856064,"bytes_avail":386354200576,"bytes_total":386547056640,"unknown_pgs_ratio":0.2800000011920929,"write_bytes_sec":1830930,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:30:03.902 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:03.898+0000 7f00f0991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:03.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:03.907+0000 7f00eb59e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:03.921 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:30:03.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:04.094 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:04.084+0000 7f710606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:04.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:04.100+0000 7f710606c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:04.418 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:04.439 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-15T12:30:04.108601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002483"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:04.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 14 16 3056 3040 13.5702 12.6875 0.0705914 0.0734581 2024-09-15T12:30:05.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 15 16 3283 3267 13.6113 14.1875 0.0615045 0.0732728 2024-09-15T12:30:06.272 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:06.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:06.448+0000 7fdef247d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:06.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:06.458+0000 7fdef247d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:06.812 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 16 16 3488 3472 13.5613 12.8125 0.0981175 0.0735558 2024-09-15T12:30:07.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:07.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:07.602+0000 7f6e78f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:07.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:07.615+0000 7f6e78f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:07.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 17 16 3703 3687 13.5539 13.4375 0.0831797 0.0735676 2024-09-15T12:30:07.942 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:07.942 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:07.960 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-15T12:30:06.109002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002461"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:07.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:30:08.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.104+0000 7fa8a2191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:08.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.115+0000 7fa8a2191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:08.565 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":88,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"unknown","count":7}],"num_pgs":25,"num_pools":3,"num_objects":321,"data_bytes":21430835,"bytes_used":192856064,"bytes_avail":386354200576,"bytes_total":386547056640,"unknown_pgs_ratio":0.2800000011920929,"write_bytes_sec":1736347,"read_op_per_sec":0,"write_op_per_sec":26},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:30:08.589 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:30:08.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:08.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:08.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.750+0000 7fdc75897640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:08.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.768+0000 7fdc75897640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:08.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 18 16 3905 3889 13.502 12.625 0.0514067 0.0739042 2024-09-15T12:30:08.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.915+0000 7ff97a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:08.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:08.928+0000 7ff97a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:09.134 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:09.134 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:09.152 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-15T12:30:08.109439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21430835,"num_objects":321,"num_object_clones":0,"num_object_copies":963,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":397,"num_write_kb":21460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":373,"ondisk_log_size":373,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":188336,"kb_used_data":5868,"kb_used_omap":0,"kb_used_meta":182400,"kb_avail":377299024,"statfs":{"total":386547056640,"available":386354200576,"internally_reserved":0,"allocated":6008832,"data_stored":49954758,"data_compressed":2323948,"data_compressed_allocated":3063808,"data_compressed_original":48775168,"omap_allocated":0,"internal_metadata":186777600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002482"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.721676+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:23.721676+0000","last_peered":"2024-09-15T12:29:23.721676+0000","last_clean":"2024-09-15T12:29:23.721676+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:23.721676+0000","last_undegraded":"2024-09-15T12:29:23.721676+0000","last_fullsized":"2024-09-15T12:29:23.721676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":220,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.712735+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:23.712735+0000","last_peered":"2024-09-15T12:29:23.712735+0000","last_clean":"2024-09-15T12:29:23.712735+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:23.712735+0000","last_undegraded":"2024-09-15T12:29:23.712735+0000","last_fullsized":"2024-09-15T12:29:23.712735+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":27,"acting":27,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59068,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57664,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1417216,"data_stored":11221128,"data_compressed":521752,"data_compressed_allocated":688128,"data_compressed_original":10928128,"omap_allocated":0,"internal_metadata":59047936},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:09.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 19 16 4116 4100 13.4854 13.1875 0.0976259 0.0739475 2024-09-15T12:30:10.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:30:10.812722+0000 min lat: 0.0159606 max lat: 0.175976 avg lat: 0.0743059 2024-09-15T12:30:10.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:30:10.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 20 16 4313 4297 13.4267 12.3125 0.0602059 0.0743059 2024-09-15T12:30:11.231 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:11.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:11.393+0000 7f879ad91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:11.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:11.401+0000 7f879ad91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:11.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 21 16 4536 4520 13.4509 13.9375 0.098577 0.074225 2024-09-15T12:30:12.154 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:12.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:12.311+0000 7fea95391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:12.327 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:12.326+0000 7fea95391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:12.669 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:12.671 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:12.690 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-15T12:30:12.110232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76153395,"num_objects":1156,"num_object_clones":0,"num_object_copies":3468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1232,"num_write_kb":74900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1208,"ondisk_log_size":1208,"up":63,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":197384,"kb_used_data":19080,"kb_used_omap":0,"kb_used_meta":178240,"kb_avail":377289976,"statfs":{"total":386547056640,"available":386344935424,"internally_reserved":0,"allocated":19537920,"data_stored":265909961,"data_compressed":12577094,"data_compressed_allocated":16560128,"data_compressed_original":264716288,"omap_allocated":0,"internal_metadata":182517760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":835,"num_write_kb":53440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002382"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":75563008,"num_objects":1153,"num_object_clones":0,"num_object_copies":3459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":73792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17235968,"data_stored":275775488,"data_compressed":13093643,"data_compressed_allocated":17235968,"data_compressed_original":275775488,"omap_allocated":0,"internal_metadata":0},"log_size":1153,"ondisk_log_size":1153,"up":36,"acting":36,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:12.691 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:30:12.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 22 16 4753 4737 13.4559 13.5625 0.0905949 0.0741647 2024-09-15T12:30:12.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:12.847+0000 7f23d70fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:12.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:12.858+0000 7f23d70fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:13.377 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:13.378 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":88,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403356,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"unknown","count":4}],"num_pgs":25,"num_pools":3,"num_objects":1156,"data_bytes":76153395,"bytes_used":202121216,"bytes_avail":386344935424,"bytes_total":386547056640,"unknown_pgs_ratio":0.15999999642372131,"write_bytes_sec":4559308,"read_op_per_sec":0,"write_op_per_sec":69},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:30:13.395 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T12:30:13.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:13.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:13.553+0000 7f8818511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:13.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:13.569+0000 7f8818511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:13.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:13.813 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 23 16 4934 4918 13.3627 11.3125 0.0918655 0.0745988 2024-09-15T12:30:13.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:13.880+0000 7f9688711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:13.891 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:13.891 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:13.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:13.890+0000 7f9688711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:13.913 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-15T12:30:12.110232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":76153395,"num_objects":1156,"num_object_clones":0,"num_object_copies":3468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":1232,"num_write_kb":74900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1208,"ondisk_log_size":1208,"up":63,"acting":63,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":197384,"kb_used_data":19080,"kb_used_omap":0,"kb_used_meta":178240,"kb_avail":377289976,"statfs":{"total":386547056640,"available":386344935424,"internally_reserved":0,"allocated":19537920,"data_stored":265909961,"data_compressed":12577094,"data_compressed_allocated":16560128,"data_compressed_original":264716288,"omap_allocated":0,"internal_metadata":182517760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":835,"num_write_kb":53440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002382"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":192,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729222+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:23.729222+0000","last_peered":"2024-09-15T12:29:23.729222+0000","last_clean":"2024-09-15T12:29:23.729222+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:23.729222+0000","last_undegraded":"2024-09-15T12:29:23.729222+0000","last_fullsized":"2024-09-15T12:29:23.729222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-15T12:29:48.698382+0000","last_change":"2024-09-15T12:29:48.698382+0000","last_active":"2024-09-15T12:29:48.698382+0000","last_peered":"2024-09-15T12:29:48.698382+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T12:29:48.698382+0000","last_undegraded":"2024-09-15T12:29:48.698382+0000","last_fullsized":"2024-09-15T12:29:48.698382+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":129,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-15T12:29:23.729891+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:23.729891+0000","last_peered":"2024-09-15T12:29:23.729891+0000","last_clean":"2024-09-15T12:29:23.729891+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:23.729891+0000","last_undegraded":"2024-09-15T12:29:23.729891+0000","last_fullsized":"2024-09-15T12:29:23.729891+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":75563008,"num_objects":1153,"num_object_clones":0,"num_object_copies":3459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":73792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17235968,"data_stored":275775488,"data_compressed":13093643,"data_compressed_allocated":17235968,"data_compressed_original":275775488,"omap_allocated":0,"internal_metadata":0},"log_size":1153,"ondisk_log_size":1153,"up":36,"acting":36,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449117,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34104,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":33344,"kb_avail":94337736,"statfs":{"total":96636764160,"available":96601841664,"internally_reserved":0,"allocated":761856,"data_stored":866397,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":34144256},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:14.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 24 16 5128 5112 13.3111 12.125 0.0822966 0.0749703 2024-09-15T12:30:15.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 25 16 5342 5326 13.3136 13.375 0.0560289 0.075017 2024-09-15T12:30:16.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:16.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:16.386+0000 7f2520874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:16.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:16.399+0000 7f2520874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:16.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 26 16 5563 5547 13.3327 13.8125 0.0486039 0.0749034 2024-09-15T12:30:16.915 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:17.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:17.064+0000 7fa570afd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:17.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:17.079+0000 7fa570afd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:17.403 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:17.403 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:17.426 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-15T12:30:16.111069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":167379507,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2624,"num_write_kb":163988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2600,"ondisk_log_size":2600,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":207808,"kb_used_data":33664,"kb_used_omap":0,"kb_used_meta":174080,"kb_avail":377279552,"statfs":{"total":386547056640,"available":386334261248,"internally_reserved":0,"allocated":34471936,"data_stored":504278500,"data_compressed":23894661,"data_compressed_allocated":31457280,"data_compressed_original":503070720,"omap_allocated":0,"internal_metadata":178257920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":145948672,"num_objects":2227,"num_object_clones":0,"num_object_copies":6681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2227,"num_write_kb":142528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002467"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"88'367","reported_seq":379,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.962372+0000","last_change":"2024-09-15T12:29:49.931440+0000","last_active":"2024-09-15T12:30:15.962372+0000","last_peered":"2024-09-15T12:30:15.962372+0000","last_clean":"2024-09-15T12:30:15.962372+0000","last_became_active":"2024-09-15T12:29:49.925846+0000","last_became_peered":"2024-09-15T12:29:49.925846+0000","last_unstale":"2024-09-15T12:30:15.962372+0000","last_undegraded":"2024-09-15T12:30:15.962372+0000","last_fullsized":"2024-09-15T12:30:15.962372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"88'318","reported_seq":330,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.877921+0000","last_change":"2024-09-15T12:29:49.884854+0000","last_active":"2024-09-15T12:30:15.877921+0000","last_peered":"2024-09-15T12:30:15.877921+0000","last_clean":"2024-09-15T12:30:15.877921+0000","last_became_active":"2024-09-15T12:29:49.877965+0000","last_became_peered":"2024-09-15T12:29:49.877965+0000","last_unstale":"2024-09-15T12:30:15.877921+0000","last_undegraded":"2024-09-15T12:30:15.877921+0000","last_fullsized":"2024-09-15T12:30:15.877921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.739124+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:50.739124+0000","last_peered":"2024-09-15T12:29:50.739124+0000","last_clean":"2024-09-15T12:29:50.739124+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:50.739124+0000","last_undegraded":"2024-09-15T12:29:50.739124+0000","last_fullsized":"2024-09-15T12:29:50.739124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"88'364","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:16.067848+0000","last_change":"2024-09-15T12:29:49.928072+0000","last_active":"2024-09-15T12:30:16.067848+0000","last_peered":"2024-09-15T12:30:16.067848+0000","last_clean":"2024-09-15T12:30:16.067848+0000","last_became_active":"2024-09-15T12:29:49.921571+0000","last_became_peered":"2024-09-15T12:29:49.921571+0000","last_unstale":"2024-09-15T12:30:16.067848+0000","last_undegraded":"2024-09-15T12:30:16.067848+0000","last_fullsized":"2024-09-15T12:30:16.067848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":364,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"88'343","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.994115+0000","last_change":"2024-09-15T12:29:49.860384+0000","last_active":"2024-09-15T12:30:15.994115+0000","last_peered":"2024-09-15T12:30:15.994115+0000","last_clean":"2024-09-15T12:30:15.994115+0000","last_became_active":"2024-09-15T12:29:49.853903+0000","last_became_peered":"2024-09-15T12:29:49.853903+0000","last_unstale":"2024-09-15T12:30:15.994115+0000","last_undegraded":"2024-09-15T12:30:15.994115+0000","last_fullsized":"2024-09-15T12:30:15.994115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737719+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:50.737719+0000","last_peered":"2024-09-15T12:29:50.737719+0000","last_clean":"2024-09-15T12:29:50.737719+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:50.737719+0000","last_undegraded":"2024-09-15T12:29:50.737719+0000","last_fullsized":"2024-09-15T12:29:50.737719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":166789120,"num_objects":2545,"num_object_clones":0,"num_object_copies":7635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2545,"num_write_kb":162880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32612352,"data_stored":521797632,"data_compressed":24775314,"data_compressed_allocated":32612352,"data_compressed_original":521797632,"omap_allocated":0,"internal_metadata":0},"log_size":2545,"ondisk_log_size":2545,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44528,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":94327312,"statfs":{"total":96636764160,"available":96591167488,"internally_reserved":0,"allocated":15695872,"data_stored":239234936,"data_compressed":11347623,"data_compressed_allocated":14938112,"data_compressed_original":238927872,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15376384,"data_stored":246022144,"data_compressed":11681671,"data_compressed_allocated":15376384,"data_compressed_original":246022144,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:17.427 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T12:30:17.427 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:30:17.428 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:17.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:17.579+0000 7fef28991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:17.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:17.592+0000 7fef28991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:17.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 27 16 5761 5745 13.2972 12.375 0.0563144 0.0750547 2024-09-15T12:30:17.931 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:17.932 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:17.951 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-15T12:30:16.111069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":167379507,"num_objects":2548,"num_object_clones":0,"num_object_copies":7644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":2624,"num_write_kb":163988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2600,"ondisk_log_size":2600,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":207808,"kb_used_data":33664,"kb_used_omap":0,"kb_used_meta":174080,"kb_avail":377279552,"statfs":{"total":386547056640,"available":386334261248,"internally_reserved":0,"allocated":34471936,"data_stored":504278500,"data_compressed":23894661,"data_compressed_allocated":31457280,"data_compressed_original":503070720,"omap_allocated":0,"internal_metadata":178257920},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":145948672,"num_objects":2227,"num_object_clones":0,"num_object_copies":6681,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2227,"num_write_kb":142528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002467"},"pg_stats":[{"pgid":"6.f","version":"88'58","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.128788+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:29:55.128788+0000","last_peered":"2024-09-15T12:29:55.128788+0000","last_clean":"2024-09-15T12:29:55.128788+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:29:55.128788+0000","last_undegraded":"2024-09-15T12:29:55.128788+0000","last_fullsized":"2024-09-15T12:29:55.128788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3801088,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58,"num_write_kb":3712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.920747+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:29:49.920747+0000","last_peered":"2024-09-15T12:29:49.920747+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:29:49.920747+0000","last_undegraded":"2024-09-15T12:29:49.920747+0000","last_fullsized":"2024-09-15T12:29:49.920747+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.893812+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:29:49.893812+0000","last_peered":"2024-09-15T12:29:49.893812+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:29:49.893812+0000","last_undegraded":"2024-09-15T12:29:49.893812+0000","last_fullsized":"2024-09-15T12:29:49.893812+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"88'367","reported_seq":379,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.962372+0000","last_change":"2024-09-15T12:29:49.931440+0000","last_active":"2024-09-15T12:30:15.962372+0000","last_peered":"2024-09-15T12:30:15.962372+0000","last_clean":"2024-09-15T12:30:15.962372+0000","last_became_active":"2024-09-15T12:29:49.925846+0000","last_became_peered":"2024-09-15T12:29:49.925846+0000","last_unstale":"2024-09-15T12:30:15.962372+0000","last_undegraded":"2024-09-15T12:30:15.962372+0000","last_fullsized":"2024-09-15T12:30:15.962372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.877502+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:29:49.877502+0000","last_peered":"2024-09-15T12:29:49.877502+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:29:49.877502+0000","last_undegraded":"2024-09-15T12:29:49.877502+0000","last_fullsized":"2024-09-15T12:29:49.877502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"88'63","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.185863+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:29:55.185863+0000","last_peered":"2024-09-15T12:29:55.185863+0000","last_clean":"2024-09-15T12:29:55.185863+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:29:55.185863+0000","last_undegraded":"2024-09-15T12:29:55.185863+0000","last_fullsized":"2024-09-15T12:29:55.185863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4128768,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"88'318","reported_seq":330,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.877921+0000","last_change":"2024-09-15T12:29:49.884854+0000","last_active":"2024-09-15T12:30:15.877921+0000","last_peered":"2024-09-15T12:30:15.877921+0000","last_clean":"2024-09-15T12:30:15.877921+0000","last_became_active":"2024-09-15T12:29:49.877965+0000","last_became_peered":"2024-09-15T12:29:49.877965+0000","last_unstale":"2024-09-15T12:30:15.877921+0000","last_undegraded":"2024-09-15T12:30:15.877921+0000","last_fullsized":"2024-09-15T12:30:15.877921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.739124+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:50.739124+0000","last_peered":"2024-09-15T12:29:50.739124+0000","last_clean":"2024-09-15T12:29:50.739124+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:50.739124+0000","last_undegraded":"2024-09-15T12:29:50.739124+0000","last_fullsized":"2024-09-15T12:29:50.739124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.742310+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:49.742310+0000","last_peered":"2024-09-15T12:29:49.742310+0000","last_clean":"2024-09-15T12:29:49.742310+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:49.742310+0000","last_undegraded":"2024-09-15T12:29:49.742310+0000","last_fullsized":"2024-09-15T12:29:49.742310+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"88'364","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:16.067848+0000","last_change":"2024-09-15T12:29:49.928072+0000","last_active":"2024-09-15T12:30:16.067848+0000","last_peered":"2024-09-15T12:30:16.067848+0000","last_clean":"2024-09-15T12:30:16.067848+0000","last_became_active":"2024-09-15T12:29:49.921571+0000","last_became_peered":"2024-09-15T12:29:49.921571+0000","last_unstale":"2024-09-15T12:30:16.067848+0000","last_undegraded":"2024-09-15T12:30:16.067848+0000","last_fullsized":"2024-09-15T12:30:16.067848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":364,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"88'66","reported_seq":78,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.262321+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:29:55.262321+0000","last_peered":"2024-09-15T12:29:55.262321+0000","last_clean":"2024-09-15T12:29:55.262321+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:29:55.262321+0000","last_undegraded":"2024-09-15T12:29:55.262321+0000","last_fullsized":"2024-09-15T12:29:55.262321+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4325376,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734117+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:49.734117+0000","last_peered":"2024-09-15T12:29:49.734117+0000","last_clean":"2024-09-15T12:29:49.734117+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:49.734117+0000","last_undegraded":"2024-09-15T12:29:49.734117+0000","last_fullsized":"2024-09-15T12:29:49.734117+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"88'343","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.994115+0000","last_change":"2024-09-15T12:29:49.860384+0000","last_active":"2024-09-15T12:30:15.994115+0000","last_peered":"2024-09-15T12:30:15.994115+0000","last_clean":"2024-09-15T12:30:15.994115+0000","last_became_active":"2024-09-15T12:29:49.853903+0000","last_became_peered":"2024-09-15T12:29:49.853903+0000","last_unstale":"2024-09-15T12:30:15.994115+0000","last_undegraded":"2024-09-15T12:30:15.994115+0000","last_fullsized":"2024-09-15T12:30:15.994115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"88'71","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.174699+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:29:55.174699+0000","last_peered":"2024-09-15T12:29:55.174699+0000","last_clean":"2024-09-15T12:29:55.174699+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:29:55.174699+0000","last_undegraded":"2024-09-15T12:29:55.174699+0000","last_fullsized":"2024-09-15T12:29:55.174699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":71,"num_object_clones":0,"num_object_copies":213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":71,"num_write_kb":4544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.741615+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:29:50.741615+0000","last_peered":"2024-09-15T12:29:50.741615+0000","last_clean":"2024-09-15T12:29:50.741615+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:29:50.741615+0000","last_undegraded":"2024-09-15T12:29:50.741615+0000","last_fullsized":"2024-09-15T12:29:50.741615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"88'60","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:55.202017+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:29:55.202017+0000","last_peered":"2024-09-15T12:29:55.202017+0000","last_clean":"2024-09-15T12:29:55.202017+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:29:55.202017+0000","last_undegraded":"2024-09-15T12:29:55.202017+0000","last_fullsized":"2024-09-15T12:29:55.202017+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3932160,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":3840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737719+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:50.737719+0000","last_peered":"2024-09-15T12:29:50.737719+0000","last_clean":"2024-09-15T12:29:50.737719+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:50.737719+0000","last_undegraded":"2024-09-15T12:29:50.737719+0000","last_fullsized":"2024-09-15T12:29:50.737719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.905664+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:29:49.905664+0000","last_peered":"2024-09-15T12:29:49.905664+0000","last_clean":"2024-09-15T12:29:48.698382+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:29:49.905664+0000","last_undegraded":"2024-09-15T12:29:49.905664+0000","last_fullsized":"2024-09-15T12:29:49.905664+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.734586+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:29:50.734586+0000","last_peered":"2024-09-15T12:29:50.734586+0000","last_clean":"2024-09-15T12:29:50.734586+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:29:50.734586+0000","last_undegraded":"2024-09-15T12:29:50.734586+0000","last_fullsized":"2024-09-15T12:29:50.734586+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T12:29:49.734620+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:49.734620+0000","last_peered":"2024-09-15T12:29:49.734620+0000","last_clean":"2024-09-15T12:29:49.734620+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:49.734620+0000","last_undegraded":"2024-09-15T12:29:49.734620+0000","last_fullsized":"2024-09-15T12:29:49.734620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":166789120,"num_objects":2545,"num_object_clones":0,"num_object_copies":7635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2545,"num_write_kb":162880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32612352,"data_stored":521797632,"data_compressed":24775314,"data_compressed_allocated":32612352,"data_compressed_original":521797632,"omap_allocated":0,"internal_metadata":0},"log_size":2545,"ondisk_log_size":2545,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44528,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":94327312,"statfs":{"total":96636764160,"available":96591167488,"internally_reserved":0,"allocated":15695872,"data_stored":239234936,"data_compressed":11347623,"data_compressed_allocated":14938112,"data_compressed_original":238927872,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57504,"kb_used_data":3024,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94314336,"statfs":{"total":96636764160,"available":96577880064,"internally_reserved":0,"allocated":3096576,"data_stored":37580683,"data_compressed":1772140,"data_compressed_allocated":2334720,"data_compressed_original":37273600,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37660,"kb_used_data":716,"kb_used_omap":0,"kb_used_meta":36928,"kb_avail":94334180,"statfs":{"total":96636764160,"available":96598200320,"internally_reserved":0,"allocated":733184,"data_stored":286550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":37814272},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2797568,"data_stored":44761088,"data_compressed":2124737,"data_compressed_allocated":2797568,"data_compressed_original":44761088,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15376384,"data_stored":246022144,"data_compressed":11681671,"data_compressed_allocated":15376384,"data_compressed_original":246022144,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:18.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:18.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 28 16 5965 5949 13.2777 12.75 0.0986857 0.0751764 2024-09-15T12:30:18.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:18.847+0000 7fcc55c9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:18.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:18.858+0000 7fcc55c9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:19.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 29 16 6179 6163 13.281 13.375 0.0589491 0.0751933 2024-09-15T12:30:20.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 30 16 6397 6381 13.2924 13.625 0.0563423 0.075133 2024-09-15T12:30:21.228 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:21.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:21.385+0000 7f5fda751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:21.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:21.393+0000 7f5fda751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:21.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 31 16 6609 6593 13.291 13.25 0.0796108 0.0751106 2024-09-15T12:30:22.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 32 16 6824 6808 13.2955 13.4375 0.0564411 0.0751281 2024-09-15T12:30:22.953 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 1, 3] 2024-09-15T12:30:22.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T12:30:23.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.117+0000 7f941fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:23.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.133+0000 7f941fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:23.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:23.756 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T12:30:23.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T12:30:23.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 33 16 7033 7017 13.2884 13.0625 0.114311 0.0750664 2024-09-15T12:30:23.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.810+0000 7fee99b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:23.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.818+0000 7fee99b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:23.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.945+0000 7f4d0d911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:23.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:23.960+0000 7f4d0d911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:24.776 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T12:30:24.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T12:30:24.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 34 16 7231 7215 13.2615 12.375 0.0788219 0.0752685 2024-09-15T12:30:24.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:24.940+0000 7f66c027f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:24.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:24.956+0000 7f66c027f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:25.777 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T12:30:25.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T12:30:25.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 35 16 7431 7415 13.2397 12.5 0.112897 0.0754407 2024-09-15T12:30:25.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:25.961+0000 7f03f10d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:25.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:25.979+0000 7f03f10d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:26.076 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:26.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:26.222+0000 7ffa2bb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:26.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:26.237+0000 7ffa2bb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:26.789 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T12:30:26.808 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T12:30:26.809 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:26.814 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 36 16 7628 7612 13.214 12.3125 0.0450435 0.0755705 2024-09-15T12:30:26.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:26.970+0000 7f8179591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:26.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:26.982+0000 7f8179591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:27.337 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:27.337 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:27.354 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T12:30:26.791687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399508019,"num_objects":6090,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":6166,"num_write_kb":390676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6142,"ondisk_log_size":6142,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":249496,"kb_used_data":71192,"kb_used_omap":0,"kb_used_meta":178240,"kb_avail":377237864,"statfs":{"total":386547056640,"available":386291572736,"internally_reserved":0,"allocated":72900608,"data_stored":1118881986,"data_compressed":53077974,"data_compressed_allocated":69869568,"data_compressed_original":1117667328,"omap_allocated":0,"internal_metadata":182517760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":129892352,"num_objects":1982,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1982,"num_write_kb":126848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.679319"},"pg_stats":[{"pgid":"6.f","version":"90'446","reported_seq":464,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.107600+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:30:25.107600+0000","last_peered":"2024-09-15T12:30:25.107600+0000","last_clean":"2024-09-15T12:30:25.107600+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:30:25.107600+0000","last_undegraded":"2024-09-15T12:30:25.107600+0000","last_fullsized":"2024-09-15T12:30:25.107600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":446,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":446,"num_write_kb":28544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"88'395","reported_seq":407,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.383241+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:30:20.383241+0000","last_peered":"2024-09-15T12:30:20.383241+0000","last_clean":"2024-09-15T12:30:20.383241+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:30:20.383241+0000","last_undegraded":"2024-09-15T12:30:20.383241+0000","last_fullsized":"2024-09-15T12:30:20.383241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":395,"num_object_clones":0,"num_object_copies":1185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":395,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"88'396","reported_seq":408,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.461881+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:30:20.461881+0000","last_peered":"2024-09-15T12:30:20.461881+0000","last_clean":"2024-09-15T12:30:20.461881+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:30:20.461881+0000","last_undegraded":"2024-09-15T12:30:20.461881+0000","last_fullsized":"2024-09-15T12:30:20.461881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25952256,"num_objects":396,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":396,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"88'367","reported_seq":379,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.962372+0000","last_change":"2024-09-15T12:29:49.931440+0000","last_active":"2024-09-15T12:30:15.962372+0000","last_peered":"2024-09-15T12:30:15.962372+0000","last_clean":"2024-09-15T12:30:15.962372+0000","last_became_active":"2024-09-15T12:29:49.925846+0000","last_became_peered":"2024-09-15T12:29:49.925846+0000","last_unstale":"2024-09-15T12:30:15.962372+0000","last_undegraded":"2024-09-15T12:30:15.962372+0000","last_fullsized":"2024-09-15T12:30:15.962372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"88'403","reported_seq":415,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.493408+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:30:20.493408+0000","last_peered":"2024-09-15T12:30:20.493408+0000","last_clean":"2024-09-15T12:30:20.493408+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:30:20.493408+0000","last_undegraded":"2024-09-15T12:30:20.493408+0000","last_fullsized":"2024-09-15T12:30:20.493408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26411008,"num_objects":403,"num_object_clones":0,"num_object_copies":1209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"90'450","reported_seq":468,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.253276+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:30:25.253276+0000","last_peered":"2024-09-15T12:30:25.253276+0000","last_clean":"2024-09-15T12:30:25.253276+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:30:25.253276+0000","last_undegraded":"2024-09-15T12:30:25.253276+0000","last_fullsized":"2024-09-15T12:30:25.253276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":450,"num_write_kb":28800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"88'318","reported_seq":330,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.877921+0000","last_change":"2024-09-15T12:29:49.884854+0000","last_active":"2024-09-15T12:30:15.877921+0000","last_peered":"2024-09-15T12:30:15.877921+0000","last_clean":"2024-09-15T12:30:15.877921+0000","last_became_active":"2024-09-15T12:29:49.877965+0000","last_became_peered":"2024-09-15T12:29:49.877965+0000","last_unstale":"2024-09-15T12:30:15.877921+0000","last_undegraded":"2024-09-15T12:30:15.877921+0000","last_fullsized":"2024-09-15T12:30:15.877921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.739124+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:50.739124+0000","last_peered":"2024-09-15T12:29:50.739124+0000","last_clean":"2024-09-15T12:29:50.739124+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:50.739124+0000","last_undegraded":"2024-09-15T12:29:50.739124+0000","last_fullsized":"2024-09-15T12:29:50.739124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.744018+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:50.744018+0000","last_peered":"2024-09-15T12:29:50.744018+0000","last_clean":"2024-09-15T12:29:50.744018+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:50.744018+0000","last_undegraded":"2024-09-15T12:29:50.744018+0000","last_fullsized":"2024-09-15T12:29:50.744018+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"88'364","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:16.067848+0000","last_change":"2024-09-15T12:29:49.928072+0000","last_active":"2024-09-15T12:30:16.067848+0000","last_peered":"2024-09-15T12:30:16.067848+0000","last_clean":"2024-09-15T12:30:16.067848+0000","last_became_active":"2024-09-15T12:29:49.921571+0000","last_became_peered":"2024-09-15T12:29:49.921571+0000","last_unstale":"2024-09-15T12:30:16.067848+0000","last_undegraded":"2024-09-15T12:30:16.067848+0000","last_fullsized":"2024-09-15T12:30:16.067848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":364,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"89'473","reported_seq":491,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.808652+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:30:24.808652+0000","last_peered":"2024-09-15T12:30:24.808652+0000","last_clean":"2024-09-15T12:30:24.808652+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:30:24.808652+0000","last_undegraded":"2024-09-15T12:30:24.808652+0000","last_fullsized":"2024-09-15T12:30:24.808652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":473,"num_object_clones":0,"num_object_copies":1419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.738652+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:50.738652+0000","last_peered":"2024-09-15T12:29:50.738652+0000","last_clean":"2024-09-15T12:29:50.738652+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:50.738652+0000","last_undegraded":"2024-09-15T12:29:50.738652+0000","last_fullsized":"2024-09-15T12:29:50.738652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"88'343","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.994115+0000","last_change":"2024-09-15T12:29:49.860384+0000","last_active":"2024-09-15T12:30:15.994115+0000","last_peered":"2024-09-15T12:30:15.994115+0000","last_clean":"2024-09-15T12:30:15.994115+0000","last_became_active":"2024-09-15T12:29:49.853903+0000","last_became_peered":"2024-09-15T12:29:49.853903+0000","last_unstale":"2024-09-15T12:30:15.994115+0000","last_undegraded":"2024-09-15T12:30:15.994115+0000","last_fullsized":"2024-09-15T12:30:15.994115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"90'478","reported_seq":496,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.118231+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:30:25.118231+0000","last_peered":"2024-09-15T12:30:25.118231+0000","last_clean":"2024-09-15T12:30:25.118231+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:30:25.118231+0000","last_undegraded":"2024-09-15T12:30:25.118231+0000","last_fullsized":"2024-09-15T12:30:25.118231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31326208,"num_objects":478,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":478,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.811658+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:30:24.811658+0000","last_peered":"2024-09-15T12:30:24.811658+0000","last_clean":"2024-09-15T12:30:24.811658+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:30:24.811658+0000","last_undegraded":"2024-09-15T12:30:24.811658+0000","last_fullsized":"2024-09-15T12:30:24.811658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"90'453","reported_seq":471,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.956988+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:30:24.956988+0000","last_peered":"2024-09-15T12:30:24.956988+0000","last_clean":"2024-09-15T12:30:24.956988+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:30:24.956988+0000","last_undegraded":"2024-09-15T12:30:24.956988+0000","last_fullsized":"2024-09-15T12:30:24.956988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":453,"num_object_clones":0,"num_object_copies":1359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":28992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737719+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:50.737719+0000","last_peered":"2024-09-15T12:29:50.737719+0000","last_clean":"2024-09-15T12:29:50.737719+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:50.737719+0000","last_undegraded":"2024-09-15T12:29:50.737719+0000","last_fullsized":"2024-09-15T12:29:50.737719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"88'366","reported_seq":378,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.545427+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:30:20.545427+0000","last_peered":"2024-09-15T12:30:20.545427+0000","last_clean":"2024-09-15T12:30:20.545427+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:30:20.545427+0000","last_undegraded":"2024-09-15T12:30:20.545427+0000","last_fullsized":"2024-09-15T12:30:20.545427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":366,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.805854+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:30:24.805854+0000","last_peered":"2024-09-15T12:30:24.805854+0000","last_clean":"2024-09-15T12:30:24.805854+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:30:24.805854+0000","last_undegraded":"2024-09-15T12:30:24.805854+0000","last_fullsized":"2024-09-15T12:30:24.805854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737107+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:50.737107+0000","last_peered":"2024-09-15T12:29:50.737107+0000","last_clean":"2024-09-15T12:29:50.737107+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:50.737107+0000","last_undegraded":"2024-09-15T12:29:50.737107+0000","last_fullsized":"2024-09-15T12:29:50.737107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":398917632,"num_objects":6087,"num_object_clones":0,"num_object_copies":18261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6087,"num_write_kb":389568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71413760,"data_stored":1142620160,"data_compressed":54254390,"data_compressed_allocated":71413760,"data_compressed_original":1142620160,"omap_allocated":0,"internal_metadata":0},"log_size":6087,"ondisk_log_size":6087,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44528,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":94327312,"statfs":{"total":96636764160,"available":96591167488,"internally_reserved":0,"allocated":15695872,"data_stored":239234936,"data_compressed":11347623,"data_compressed_allocated":14938112,"data_compressed_original":238927872,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74936,"kb_used_data":20456,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94296904,"statfs":{"total":96636764160,"available":96560029696,"internally_reserved":0,"allocated":20946944,"data_stored":323058772,"data_compressed":15327867,"data_compressed_allocated":20176896,"data_compressed_original":322748416,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61916,"kb_used_data":20812,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94309924,"statfs":{"total":96636764160,"available":96573362176,"internally_reserved":0,"allocated":21311488,"data_stored":329411947,"data_compressed":15627586,"data_compressed_allocated":20570112,"data_compressed_original":329121792,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21024768,"data_stored":336396288,"data_compressed":15973018,"data_compressed_allocated":21024768,"data_compressed_original":336396288,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20574208,"data_stored":329187328,"data_compressed":15630795,"data_compressed_allocated":20574208,"data_compressed_original":329187328,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15376384,"data_stored":246022144,"data_compressed":11681671,"data_compressed_allocated":15376384,"data_compressed_original":246022144,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:27.356 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:30:27.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:27.563+0000 7f94900fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:27.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:27.577+0000 7f94900fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:27.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 37 16 7807 7791 13.1592 11.1875 0.0750316 0.075907 2024-09-15T12:30:27.890 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:30:27.890 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:30:27.913 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T12:30:26.791687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399508019,"num_objects":6090,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":6166,"num_write_kb":390676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6142,"ondisk_log_size":6142,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":249496,"kb_used_data":71192,"kb_used_omap":0,"kb_used_meta":178240,"kb_avail":377237864,"statfs":{"total":386547056640,"available":386291572736,"internally_reserved":0,"allocated":72900608,"data_stored":1118881986,"data_compressed":53077974,"data_compressed_allocated":69869568,"data_compressed_original":1117667328,"omap_allocated":0,"internal_metadata":182517760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":129892352,"num_objects":1982,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1982,"num_write_kb":126848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.679319"},"pg_stats":[{"pgid":"6.f","version":"90'446","reported_seq":464,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.107600+0000","last_change":"2024-09-15T12:29:49.883834+0000","last_active":"2024-09-15T12:30:25.107600+0000","last_peered":"2024-09-15T12:30:25.107600+0000","last_clean":"2024-09-15T12:30:25.107600+0000","last_became_active":"2024-09-15T12:29:49.875229+0000","last_became_peered":"2024-09-15T12:29:49.875229+0000","last_unstale":"2024-09-15T12:30:25.107600+0000","last_undegraded":"2024-09-15T12:30:25.107600+0000","last_fullsized":"2024-09-15T12:30:25.107600+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29229056,"num_objects":446,"num_object_clones":0,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":446,"num_write_kb":28544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"88'395","reported_seq":407,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.383241+0000","last_change":"2024-09-15T12:29:49.922718+0000","last_active":"2024-09-15T12:30:20.383241+0000","last_peered":"2024-09-15T12:30:20.383241+0000","last_clean":"2024-09-15T12:30:20.383241+0000","last_became_active":"2024-09-15T12:29:49.913716+0000","last_became_peered":"2024-09-15T12:29:49.913716+0000","last_unstale":"2024-09-15T12:30:20.383241+0000","last_undegraded":"2024-09-15T12:30:20.383241+0000","last_fullsized":"2024-09-15T12:30:20.383241+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25886720,"num_objects":395,"num_object_clones":0,"num_object_copies":1185,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":395,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"88'396","reported_seq":408,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.461881+0000","last_change":"2024-09-15T12:29:49.895926+0000","last_active":"2024-09-15T12:30:20.461881+0000","last_peered":"2024-09-15T12:30:20.461881+0000","last_clean":"2024-09-15T12:30:20.461881+0000","last_became_active":"2024-09-15T12:29:49.881988+0000","last_became_peered":"2024-09-15T12:29:49.881988+0000","last_unstale":"2024-09-15T12:30:20.461881+0000","last_undegraded":"2024-09-15T12:30:20.461881+0000","last_fullsized":"2024-09-15T12:30:20.461881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25952256,"num_objects":396,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":396,"num_write_kb":25344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"88'367","reported_seq":379,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.962372+0000","last_change":"2024-09-15T12:29:49.931440+0000","last_active":"2024-09-15T12:30:15.962372+0000","last_peered":"2024-09-15T12:30:15.962372+0000","last_clean":"2024-09-15T12:30:15.962372+0000","last_became_active":"2024-09-15T12:29:49.925846+0000","last_became_peered":"2024-09-15T12:29:49.925846+0000","last_unstale":"2024-09-15T12:30:15.962372+0000","last_undegraded":"2024-09-15T12:30:15.962372+0000","last_fullsized":"2024-09-15T12:30:15.962372+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24051712,"num_objects":367,"num_object_clones":0,"num_object_copies":1101,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":23488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"88'403","reported_seq":415,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.493408+0000","last_change":"2024-09-15T12:29:49.878835+0000","last_active":"2024-09-15T12:30:20.493408+0000","last_peered":"2024-09-15T12:30:20.493408+0000","last_clean":"2024-09-15T12:30:20.493408+0000","last_became_active":"2024-09-15T12:29:49.863367+0000","last_became_peered":"2024-09-15T12:29:49.863367+0000","last_unstale":"2024-09-15T12:30:20.493408+0000","last_undegraded":"2024-09-15T12:30:20.493408+0000","last_fullsized":"2024-09-15T12:30:20.493408+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26411008,"num_objects":403,"num_object_clones":0,"num_object_copies":1209,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":403,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"90'450","reported_seq":468,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.253276+0000","last_change":"2024-09-15T12:29:49.836926+0000","last_active":"2024-09-15T12:30:25.253276+0000","last_peered":"2024-09-15T12:30:25.253276+0000","last_clean":"2024-09-15T12:30:25.253276+0000","last_became_active":"2024-09-15T12:29:49.828482+0000","last_became_peered":"2024-09-15T12:29:49.828482+0000","last_unstale":"2024-09-15T12:30:25.253276+0000","last_undegraded":"2024-09-15T12:30:25.253276+0000","last_fullsized":"2024-09-15T12:30:25.253276+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29491200,"num_objects":450,"num_object_clones":0,"num_object_copies":1350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":450,"num_write_kb":28800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"88'318","reported_seq":330,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.877921+0000","last_change":"2024-09-15T12:29:49.884854+0000","last_active":"2024-09-15T12:30:15.877921+0000","last_peered":"2024-09-15T12:30:15.877921+0000","last_clean":"2024-09-15T12:30:15.877921+0000","last_became_active":"2024-09-15T12:29:49.877965+0000","last_became_peered":"2024-09-15T12:29:49.877965+0000","last_unstale":"2024-09-15T12:30:15.877921+0000","last_undegraded":"2024-09-15T12:30:15.877921+0000","last_fullsized":"2024-09-15T12:30:15.877921+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20840448,"num_objects":318,"num_object_clones":0,"num_object_copies":954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":318,"num_write_kb":20352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.8","version":"88'273","reported_seq":285,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.959572+0000","last_change":"2024-09-15T12:29:49.842346+0000","last_active":"2024-09-15T12:30:10.959572+0000","last_peered":"2024-09-15T12:30:10.959572+0000","last_clean":"2024-09-15T12:30:10.959572+0000","last_became_active":"2024-09-15T12:29:49.824959+0000","last_became_peered":"2024-09-15T12:29:49.824959+0000","last_unstale":"2024-09-15T12:30:10.959572+0000","last_undegraded":"2024-09-15T12:30:10.959572+0000","last_fullsized":"2024-09-15T12:30:10.959572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891328,"num_objects":273,"num_object_clones":0,"num_object_copies":819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":17472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"88'292","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.924262+0000","last_change":"2024-09-15T12:29:49.915848+0000","last_active":"2024-09-15T12:30:10.924262+0000","last_peered":"2024-09-15T12:30:10.924262+0000","last_clean":"2024-09-15T12:30:10.924262+0000","last_became_active":"2024-09-15T12:29:49.899083+0000","last_became_peered":"2024-09-15T12:29:49.899083+0000","last_unstale":"2024-09-15T12:30:10.924262+0000","last_undegraded":"2024-09-15T12:30:10.924262+0000","last_fullsized":"2024-09-15T12:30:10.924262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19136512,"num_objects":292,"num_object_clones":0,"num_object_copies":876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":292,"num_write_kb":18688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":204,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.739124+0000","last_change":"2024-09-15T12:29:17.419309+0000","last_active":"2024-09-15T12:29:50.739124+0000","last_peered":"2024-09-15T12:29:50.739124+0000","last_clean":"2024-09-15T12:29:50.739124+0000","last_became_active":"2024-09-15T12:29:17.407092+0000","last_became_peered":"2024-09-15T12:29:17.407092+0000","last_unstale":"2024-09-15T12:29:50.739124+0000","last_undegraded":"2024-09-15T12:29:50.739124+0000","last_fullsized":"2024-09-15T12:29:50.739124+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.744018+0000","last_change":"2024-09-15T12:27:52.742262+0000","last_active":"2024-09-15T12:29:50.744018+0000","last_peered":"2024-09-15T12:29:50.744018+0000","last_clean":"2024-09-15T12:29:50.744018+0000","last_became_active":"2024-09-15T12:27:51.864454+0000","last_became_peered":"2024-09-15T12:27:51.864454+0000","last_unstale":"2024-09-15T12:29:50.744018+0000","last_undegraded":"2024-09-15T12:29:50.744018+0000","last_fullsized":"2024-09-15T12:29:50.744018+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"88'364","reported_seq":376,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:16.067848+0000","last_change":"2024-09-15T12:29:49.928072+0000","last_active":"2024-09-15T12:30:16.067848+0000","last_peered":"2024-09-15T12:30:16.067848+0000","last_clean":"2024-09-15T12:30:16.067848+0000","last_became_active":"2024-09-15T12:29:49.921571+0000","last_became_peered":"2024-09-15T12:29:49.921571+0000","last_unstale":"2024-09-15T12:30:16.067848+0000","last_undegraded":"2024-09-15T12:30:16.067848+0000","last_fullsized":"2024-09-15T12:30:16.067848+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855104,"num_objects":364,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":364,"num_write_kb":23296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":230,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.745051+0000","last_change":"2024-09-15T12:27:52.758732+0000","last_active":"2024-09-15T12:29:50.745051+0000","last_peered":"2024-09-15T12:29:50.745051+0000","last_clean":"2024-09-15T12:29:50.745051+0000","last_became_active":"2024-09-15T12:27:51.872174+0000","last_became_peered":"2024-09-15T12:27:51.872174+0000","last_unstale":"2024-09-15T12:29:50.745051+0000","last_undegraded":"2024-09-15T12:29:50.745051+0000","last_fullsized":"2024-09-15T12:29:50.745051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"89'473","reported_seq":491,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.808652+0000","last_change":"2024-09-15T12:29:49.846118+0000","last_active":"2024-09-15T12:30:24.808652+0000","last_peered":"2024-09-15T12:30:24.808652+0000","last_clean":"2024-09-15T12:30:24.808652+0000","last_became_active":"2024-09-15T12:29:49.841451+0000","last_became_peered":"2024-09-15T12:29:49.841451+0000","last_unstale":"2024-09-15T12:30:24.808652+0000","last_undegraded":"2024-09-15T12:30:24.808652+0000","last_fullsized":"2024-09-15T12:30:24.808652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30998528,"num_objects":473,"num_object_clones":0,"num_object_copies":1419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.738652+0000","last_change":"2024-09-15T12:29:18.219017+0000","last_active":"2024-09-15T12:29:50.738652+0000","last_peered":"2024-09-15T12:29:50.738652+0000","last_clean":"2024-09-15T12:29:50.738652+0000","last_became_active":"2024-09-15T12:29:17.282464+0000","last_became_peered":"2024-09-15T12:29:17.282464+0000","last_unstale":"2024-09-15T12:29:50.738652+0000","last_undegraded":"2024-09-15T12:29:50.738652+0000","last_fullsized":"2024-09-15T12:29:50.738652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"88'343","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:15.994115+0000","last_change":"2024-09-15T12:29:49.860384+0000","last_active":"2024-09-15T12:30:15.994115+0000","last_peered":"2024-09-15T12:30:15.994115+0000","last_clean":"2024-09-15T12:30:15.994115+0000","last_became_active":"2024-09-15T12:29:49.853903+0000","last_became_peered":"2024-09-15T12:29:49.853903+0000","last_unstale":"2024-09-15T12:30:15.994115+0000","last_undegraded":"2024-09-15T12:30:15.994115+0000","last_fullsized":"2024-09-15T12:30:15.994115+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":232,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.736531+0000","last_change":"2024-09-15T12:27:52.754269+0000","last_active":"2024-09-15T12:29:50.736531+0000","last_peered":"2024-09-15T12:29:50.736531+0000","last_clean":"2024-09-15T12:29:50.736531+0000","last_became_active":"2024-09-15T12:27:51.821534+0000","last_became_peered":"2024-09-15T12:27:51.821534+0000","last_unstale":"2024-09-15T12:29:50.736531+0000","last_undegraded":"2024-09-15T12:29:50.736531+0000","last_fullsized":"2024-09-15T12:29:50.736531+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"90'478","reported_seq":496,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:25.118231+0000","last_change":"2024-09-15T12:29:49.861438+0000","last_active":"2024-09-15T12:30:25.118231+0000","last_peered":"2024-09-15T12:30:25.118231+0000","last_clean":"2024-09-15T12:30:25.118231+0000","last_became_active":"2024-09-15T12:29:49.855926+0000","last_became_peered":"2024-09-15T12:29:49.855926+0000","last_unstale":"2024-09-15T12:30:25.118231+0000","last_undegraded":"2024-09-15T12:30:25.118231+0000","last_fullsized":"2024-09-15T12:30:25.118231+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31326208,"num_objects":478,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":478,"num_write_kb":30592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.811658+0000","last_change":"2024-09-15T12:29:18.223488+0000","last_active":"2024-09-15T12:30:24.811658+0000","last_peered":"2024-09-15T12:30:24.811658+0000","last_clean":"2024-09-15T12:30:24.811658+0000","last_became_active":"2024-09-15T12:29:17.360795+0000","last_became_peered":"2024-09-15T12:29:17.360795+0000","last_unstale":"2024-09-15T12:30:24.811658+0000","last_undegraded":"2024-09-15T12:30:24.811658+0000","last_fullsized":"2024-09-15T12:30:24.811658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"90'453","reported_seq":471,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.956988+0000","last_change":"2024-09-15T12:29:49.870050+0000","last_active":"2024-09-15T12:30:24.956988+0000","last_peered":"2024-09-15T12:30:24.956988+0000","last_clean":"2024-09-15T12:30:24.956988+0000","last_became_active":"2024-09-15T12:29:49.865319+0000","last_became_peered":"2024-09-15T12:29:49.865319+0000","last_unstale":"2024-09-15T12:30:24.956988+0000","last_undegraded":"2024-09-15T12:30:24.956988+0000","last_fullsized":"2024-09-15T12:30:24.956988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":453,"num_object_clones":0,"num_object_copies":1359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":453,"num_write_kb":28992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":141,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737719+0000","last_change":"2024-09-15T12:29:18.227472+0000","last_active":"2024-09-15T12:29:50.737719+0000","last_peered":"2024-09-15T12:29:50.737719+0000","last_clean":"2024-09-15T12:29:50.737719+0000","last_became_active":"2024-09-15T12:29:17.357660+0000","last_became_peered":"2024-09-15T12:29:17.357660+0000","last_unstale":"2024-09-15T12:29:50.737719+0000","last_undegraded":"2024-09-15T12:29:50.737719+0000","last_fullsized":"2024-09-15T12:29:50.737719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"88'366","reported_seq":378,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:20.545427+0000","last_change":"2024-09-15T12:29:49.907631+0000","last_active":"2024-09-15T12:30:20.545427+0000","last_peered":"2024-09-15T12:30:20.545427+0000","last_clean":"2024-09-15T12:30:20.545427+0000","last_became_active":"2024-09-15T12:29:49.889551+0000","last_became_peered":"2024-09-15T12:29:49.889551+0000","last_unstale":"2024-09-15T12:30:20.545427+0000","last_undegraded":"2024-09-15T12:30:20.545427+0000","last_fullsized":"2024-09-15T12:30:20.545427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986176,"num_objects":366,"num_object_clones":0,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":366,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-15T12:30:24.805854+0000","last_change":"2024-09-15T12:29:18.229630+0000","last_active":"2024-09-15T12:30:24.805854+0000","last_peered":"2024-09-15T12:30:24.805854+0000","last_clean":"2024-09-15T12:30:24.805854+0000","last_became_active":"2024-09-15T12:29:17.315672+0000","last_became_peered":"2024-09-15T12:29:17.315672+0000","last_unstale":"2024-09-15T12:30:24.805854+0000","last_undegraded":"2024-09-15T12:30:24.805854+0000","last_fullsized":"2024-09-15T12:30:24.805854+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"88'270","reported_seq":282,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:30:10.914123+0000","last_change":"2024-09-15T12:29:49.908081+0000","last_active":"2024-09-15T12:30:10.914123+0000","last_peered":"2024-09-15T12:30:10.914123+0000","last_clean":"2024-09-15T12:30:10.914123+0000","last_became_active":"2024-09-15T12:29:49.892548+0000","last_became_peered":"2024-09-15T12:29:49.892548+0000","last_unstale":"2024-09-15T12:30:10.914123+0000","last_undegraded":"2024-09-15T12:30:10.914123+0000","last_fullsized":"2024-09-15T12:30:10.914123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17694720,"num_objects":270,"num_object_clones":0,"num_object_copies":810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":270,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T12:29:50.737107+0000","last_change":"2024-09-15T12:27:52.738555+0000","last_active":"2024-09-15T12:29:50.737107+0000","last_peered":"2024-09-15T12:29:50.737107+0000","last_clean":"2024-09-15T12:29:50.737107+0000","last_became_active":"2024-09-15T12:27:51.831155+0000","last_became_peered":"2024-09-15T12:27:51.831155+0000","last_unstale":"2024-09-15T12:29:50.737107+0000","last_undegraded":"2024-09-15T12:29:50.737107+0000","last_fullsized":"2024-09-15T12:29:50.737107+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":398917632,"num_objects":6087,"num_object_clones":0,"num_object_copies":18261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6087,"num_write_kb":389568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":71413760,"data_stored":1142620160,"data_compressed":54254390,"data_compressed_allocated":71413760,"data_compressed_original":1142620160,"omap_allocated":0,"internal_metadata":0},"log_size":6087,"ondisk_log_size":6087,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":78,"seq":335007449147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44528,"kb_used_data":15328,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":94327312,"statfs":{"total":96636764160,"available":96591167488,"internally_reserved":0,"allocated":15695872,"data_stored":239234936,"data_compressed":11347623,"data_compressed_allocated":14938112,"data_compressed_original":238927872,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":53,"seq":227633266896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74936,"kb_used_data":20456,"kb_used_omap":0,"kb_used_meta":54464,"kb_avail":94296904,"statfs":{"total":96636764160,"available":96560029696,"internally_reserved":0,"allocated":20946944,"data_stored":323058772,"data_compressed":15327867,"data_compressed_allocated":20176896,"data_compressed_original":322748416,"omap_allocated":0,"internal_metadata":55771136},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61916,"kb_used_data":20812,"kb_used_omap":0,"kb_used_meta":41088,"kb_avail":94309924,"statfs":{"total":96636764160,"available":96573362176,"internally_reserved":0,"allocated":21311488,"data_stored":329411947,"data_compressed":15627586,"data_compressed_allocated":20570112,"data_compressed_original":329121792,"omap_allocated":0,"internal_metadata":42074112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":68116,"kb_used_data":14596,"kb_used_omap":0,"kb_used_meta":53504,"kb_avail":94303724,"statfs":{"total":96636764160,"available":96567013376,"internally_reserved":0,"allocated":14946304,"data_stored":227176331,"data_compressed":10774898,"data_compressed_allocated":14184448,"data_compressed_original":226869248,"omap_allocated":0,"internal_metadata":54788096},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":231014400,"data_compressed":10968906,"data_compressed_allocated":14438400,"data_compressed_original":231014400,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21024768,"data_stored":336396288,"data_compressed":15973018,"data_compressed_allocated":21024768,"data_compressed_original":336396288,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20574208,"data_stored":329187328,"data_compressed":15630795,"data_compressed_allocated":20574208,"data_compressed_original":329187328,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15376384,"data_stored":246022144,"data_compressed":11681671,"data_compressed_allocated":15376384,"data_compressed_original":246022144,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:30:27.915 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T12:30:27.915 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:30:27.915 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 2, 1, 3] 2024-09-15T12:30:27.915 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:30:27.916 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:30:28.007 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:30:28.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:28.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:28.795+0000 7f41e7991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:28.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:28.801+0000 7f41e7991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:28.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 38 16 7840 7824 12.8671 2.0625 0.0186954 0.0758823 2024-09-15T12:30:29.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 39 16 7840 7824 12.5372 0 - 0.0758823 2024-09-15T12:30:30.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:30:30.814585+0000 min lat: 0.0159606 max lat: 0.200095 avg lat: 0.0758823 2024-09-15T12:30:30.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:30:30.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 40 16 7840 7824 12.2238 0 - 0.0758823 2024-09-15T12:30:31.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 41 16 7840 7824 11.9256 0 - 0.0758823 2024-09-15T12:30:32.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 42 16 7840 7824 11.6417 0 - 0.0758823 2024-09-15T12:30:33.008 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:30:33.008 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:30:33.008 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:30:33.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:30:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.123+0000 7fcb038d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.132+0000 7fcb038d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:30:33.151 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:30:33.152 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:30:33.152 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:30:33.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:30:33.154 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:30:33.155 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:30:33.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.256+0000 7fa5bbf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.265+0000 7fa5bbf51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:30:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:30:33.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:30:33.285 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:30:33.285 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:30:33.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.386+0000 7f6dfdf11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.395+0000 7f6dfdf11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.413 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:30:33.413 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:30:33.413 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:30:33.413 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:30:33.413 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:30:33.414 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:30:33.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:30:33.417 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:30:33.417 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:30:33.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.519+0000 7ff0df391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:33.527+0000 7ff0df391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:33.815 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 43 16 7840 7824 11.3709 0 - 0.0758823 2024-09-15T12:30:34.816 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 44 16 7840 7824 11.1125 0 - 0.0758823 2024-09-15T12:30:35.816 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 45 16 7840 7824 10.8655 0 - 0.0758823 2024-09-15T12:30:36.816 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 46 16 7840 7824 10.6293 0 - 0.0758823 2024-09-15T12:30:37.816 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 47 16 7840 7824 10.4032 0 - 0.0758823 2024-09-15T12:30:38.816 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 48 16 7840 7824 10.1864 0 - 0.0758823 2024-09-15T12:30:39.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 49 16 7840 7824 9.97853 0 - 0.0758823 2024-09-15T12:30:40.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 50 16 7840 7824 9.77895 0 - 0.0758823 2024-09-15T12:30:41.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 51 16 7840 7824 9.5872 0 - 0.0758823 2024-09-15T12:30:42.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 52 16 7840 7824 9.40282 0 - 0.0758823 2024-09-15T12:30:43.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 53 16 7840 7824 9.22539 0 - 0.0758823 2024-09-15T12:30:44.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 54 16 7840 7824 9.05455 0 - 0.0758823 2024-09-15T12:30:45.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 55 16 7840 7824 8.88991 0 - 0.0758823 2024-09-15T12:30:46.817 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 56 16 7840 7824 8.73116 0 - 0.0758823 2024-09-15T12:30:47.818 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 57 16 7840 7824 8.57798 0 - 0.0758823 2024-09-15T12:30:48.818 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 58 16 7840 7824 8.43007 0 - 0.0758823 2024-09-15T12:30:49.818 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 59 16 7840 7824 8.28718 0 - 0.0758823 2024-09-15T12:30:50.818 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:30:50.817551+0000 min lat: 0.0159606 max lat: 0.200095 avg lat: 0.0758823 2024-09-15T12:30:50.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:30:50.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 60 16 7840 7824 8.14905 0 - 0.0758823 2024-09-15T12:30:51.818 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 61 16 7840 7824 8.01545 0 - 0.0758823 2024-09-15T12:30:52.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 62 16 7840 7824 7.88617 0 - 0.0758823 2024-09-15T12:30:53.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 63 16 7840 7824 7.76098 0 - 0.0758823 2024-09-15T12:30:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:30:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:30:54.739 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:30:54.739 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:30:54.740 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:30:54.740 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:30:54.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 64 16 7840 7824 7.63972 0 - 0.0758823 2024-09-15T12:30:55.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 65 16 7841 7825 7.52315 0.00231481 27.8925 0.0794371 2024-09-15T12:30:56.741 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:30:56.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 66 16 8025 8009 7.58339 11.5 0.121091 0.131756 2024-09-15T12:30:56.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:56.909+0000 7f96888fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:56.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:56.921+0000 7f96888fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:57.177 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:30:57.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:30:57.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:30:57.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 67 16 8210 8194 7.64276 11.5625 0.0969069 0.13076 2024-09-15T12:30:58.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 68 16 8381 8365 7.68752 10.6875 0.134903 0.12997 2024-09-15T12:30:59.183 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:30:59.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:59.342+0000 7f8e9d8fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:59.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:59.354+0000 7f8e9d8fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:59.581 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:30:59.586 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:30:59.587 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:30:59.740 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:30:59.740 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:30:59.740 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:30:59.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:30:59.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 69 16 8573 8557 7.75 12 0.0285966 0.128908 2024-09-15T12:30:59.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:59.900+0000 7ff91d351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:59.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:30:59.918+0000 7ff91d351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:30:59.962 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:30:59.962 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:30:59.962 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:30:59.962 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:30:59.962 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:30:59.963 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:30:59.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:30:59.966 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:30:59.966 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:31:00.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.125+0000 7fa42cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.138+0000 7fa42cb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:31:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:31:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:31:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:31:00.200 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:31:00.201 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:31:00.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:00.205 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:00.205 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:31:00.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.358+0000 7f8045c74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.376+0000 7f8045c74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:31:00.405 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:31:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:31:00.409 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:00.409 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:00.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:31:00.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.547+0000 7f95f9f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:00.561+0000 7f95f9f11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:00.570 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:00.573 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:00.573 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:00.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 70 16 8750 8734 7.79731 11.0625 0.111227 0.128146 2024-09-15T12:31:01.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:01.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:01.745+0000 7f0975e74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:01.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:01.755+0000 7f0975e74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:01.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 71 16 8942 8926 7.85648 12 0.0833596 0.127195 2024-09-15T12:31:02.024 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:02.028 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:02.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:02.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 72 16 9128 9112 7.90881 11.625 0.0661905 0.126374 2024-09-15T12:31:03.819 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 73 16 9322 9306 7.96655 12.125 0.0752656 0.125434 2024-09-15T12:31:04.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:04.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:04.189+0000 7ffb57b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:04.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:04.200+0000 7ffb57b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:04.399 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:04.403 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:04.403 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:04.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 74 16 9505 9489 8.01344 11.4375 0.0444263 0.124709 2024-09-15T12:31:05.575 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:31:05.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:31:05.575 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:31:05.575 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:31:05.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:05.722+0000 7f531386c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:05.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:05.736+0000 7f531386c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:31:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:31:05.788 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:31:05.789 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:31:05.790 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:31:05.794 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:05.794 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:05.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:31:05.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 75 16 9680 9664 8.05241 10.9375 0.0387822 0.124117 2024-09-15T12:31:05.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:05.947+0000 7fa820311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:05.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:05.968+0000 7fa820311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.143 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:31:06.143 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:31:06.143 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:31:06.144 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:31:06.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:06.147 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:06.148 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:31:06.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.349+0000 7f7ee5391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.365+0000 7f7ee5391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.404 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:31:06.407 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:31:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:31:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:31:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:31:06.408 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:31:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:06.412 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:06.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:31:06.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.564+0000 7fe528a7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.581+0000 7fe528a7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.623+0000 7f929a6ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:06.642+0000 7f929a6ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:06.654 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:06.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:06.659 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:31:06.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 76 16 9854 9838 8.08954 10.875 0.0274051 0.123501 2024-09-15T12:31:06.915 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:06.919 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:06.919 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:07.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 77 16 10041 10025 8.13625 11.6875 0.0219409 0.12282 2024-09-15T12:31:08.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 78 16 10229 10213 8.18256 11.75 0.129314 0.122132 2024-09-15T12:31:08.921 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:09.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:09.071+0000 7fc231f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:09.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:09.080+0000 7fc231f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:09.309 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:09.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:09.314 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:09.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 79 16 10407 10391 8.2198 11.125 0.0628053 0.121579 2024-09-15T12:31:10.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:31:10.819605+0000 min lat: 0.0159606 max lat: 28.0619 avg lat: 0.120881 2024-09-15T12:31:10.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:31:10.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 80 16 10599 10583 8.26703 12 0.0370291 0.120881 2024-09-15T12:31:11.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:11.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:11.460+0000 7f8adfc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:11.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:11.469+0000 7f8adfc4f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:11.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 2, 1] 2024-09-15T12:31:11.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:31:11.660 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 2, 1] 2024-09-15T12:31:11.660 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T12:31:11.660 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:31:11.744 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T12:31:11.820 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 81 16 10773 10757 8.29922 10.875 0.0756958 0.120347 2024-09-15T12:31:11.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T12:31:12.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 82 16 10777 10761 8.20105 0.25 0.0508723 0.12032 2024-09-15T12:31:13.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 83 16 10777 10761 8.10224 0 - 0.12032 2024-09-15T12:31:14.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 84 16 10777 10761 8.00578 0 - 0.12032 2024-09-15T12:31:15.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 85 16 10777 10761 7.91159 0 - 0.12032 2024-09-15T12:31:16.746 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3, 2] live_osds: [0, 1] 2024-09-15T12:31:16.746 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:31:16.746 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-15T12:31:16.746 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T12:31:16.746 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-15T12:31:16.749 INFO:tasks.ceph.osd.2:Started 2024-09-15T12:31:16.749 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:31:16.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 86 16 10777 10761 7.8196 0 - 0.12032 2024-09-15T12:31:16.860 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:31:16.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:16.864 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:31:17.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-15T12:31:17.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 87 16 10777 10761 7.72972 0 - 0.12032 2024-09-15T12:31:18.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 88 16 10777 10761 7.64188 0 - 0.12032 2024-09-15T12:31:19.821 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 89 16 10777 10761 7.55601 0 - 0.12032 2024-09-15T12:31:20.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 90 16 10777 10761 7.47205 0 - 0.12032 2024-09-15T12:31:21.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 91 16 10777 10761 7.38994 0 - 0.12032 2024-09-15T12:31:21.865 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:31:21.961 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:31:21.964 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:21.965 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T12:31:22.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 92 16 10777 10761 7.30962 0 - 0.12032 2024-09-15T12:31:23.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 93 16 10777 10761 7.23101 0 - 0.12032 2024-09-15T12:31:24.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 94 16 10777 10761 7.15408 0 - 0.12032 2024-09-15T12:31:25.725 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-15T12:31:25.726 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:25.731 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:25.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:25.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 95 16 10777 10761 7.07878 0 - 0.12032 2024-09-15T12:31:26.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 96 16 10777 10761 7.00505 0 - 0.12032 2024-09-15T12:31:26.966 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T12:31:27.071 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [ 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 29, 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.731971+0000" 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.732004+0000" 2024-09-15T12:31:27.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.732230+0000" 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.732432+0000" 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.792882+0000" 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.074 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.794540+0000" 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.795030+0000" 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:27.077 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.795228+0000" 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.795333+0000", 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "6.5" 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.078 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.079 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:27.081 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 37, 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.084 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.748450+0000" 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.748469+0000" 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.755494+0000" 2024-09-15T12:31:27.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:27.086 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.794055+0000" 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.796368+0000" 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.087 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.796729+0000" 2024-09-15T12:31:27.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:27.089 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.796925+0000" 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797016+0000", 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:31:27.090 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:31:27.092 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.092 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "6.a" 2024-09-15T12:31:27.092 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.092 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.094 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.095 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.096 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.097 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 45, 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.791290+0000" 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.791311+0000" 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.794696+0000" 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.099 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.795039+0000" 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797431+0000" 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.101 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797820+0000" 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.798040+0000", 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 37 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.105 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.106 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:27.107 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.109 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 48, 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797247+0000" 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797257+0000" 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.110 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797713+0000" 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.111 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.797969+0000" 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.801304+0000" 2024-09-15T12:31:27.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.801745+0000" 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:26.801967+0000", 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "blocker": { 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "waiting": 1, 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: "held_by_operation_id": 29 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.115 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:27.116 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.119 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:31:27.121 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:31:27.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:27.822 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 97 16 10777 10761 6.93283 0 - 0.12032 2024-09-15T12:31:27.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:27.873+0000 7fcbb0e88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:27.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:27.884+0000 7fcbb0e88640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:28.098 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:28.103 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:28.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total time run: 97.2702 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total writes made: 10777 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Write size: 65536 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object size: 65536 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Bandwidth (MB/sec): 6.92465 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Bandwidth: 6.35543 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max bandwidth (MB/sec): 14.5 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min bandwidth (MB/sec): 0 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average IOPS: 110 2024-09-15T12:31:28.823 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev IOPS: 101.687 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max IOPS: 232 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min IOPS: 0 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average Latency(s): 0.144318 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Latency(s): 1.24122 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max latency(s): 28.0619 2024-09-15T12:31:28.824 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min latency(s): 0.0159606 2024-09-15T12:31:28.851 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T12:31:30.105 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:30.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:30.253+0000 7fa3ef351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:30.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:30.262+0000 7fa3ef351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:30.495 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:30.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:30.501 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:32.123 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2] 2024-09-15T12:31:32.123 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:31:32.123 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-15T12:31:32.123 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T12:31:32.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2024-09-15T12:31:32.126 INFO:tasks.ceph.osd.3:Started 2024-09-15T12:31:32.126 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:31:32.248 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:31:32.251 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:32.251 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:31:32.502 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:32.645+0000 7fdafdd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:32.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:32.654+0000 7fdafdd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:33.034 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:33.037 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:33.038 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:35.039 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:35.198 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:35.195+0000 7f5c05951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:35.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:35.203+0000 7f5c05951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:35.421 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:35.425 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:35.425 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:37.252 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:31:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:31:37.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:37.373 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:31:37.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:37.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:37.589+0000 7f3088391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:37.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:37.597+0000 7f3088391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:37.848 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:37.852 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:37.852 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:39.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:40.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:40.009+0000 7f3983a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:40.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:40.017+0000 7f3983a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:40.253 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:40.258 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:42.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:42.374 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:31:42.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:42.405+0000 7f5c7cd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:42.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:42.415+0000 7f5c7cd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:42.513 INFO:teuthology.orchestra.run.smithi117.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:31:42.516 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:31:42.517 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-15T12:31:42.694 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-15T12:31:42.698 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:31:42.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:31:44.699 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:44.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:44.850+0000 7fe812ad1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:44.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:44.857+0000 7fe812ad1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:47.120 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:47.226+0000 7f8a57d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:47.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:47.232+0000 7f8a57d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:47.518 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T12:31:47.703 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-15T12:31:47.703 INFO:teuthology.orchestra.run.smithi117.stdout: "ops_in_flight": [ 2024-09-15T12:31:47.703 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.703 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "replicated_request", 2024-09-15T12:31:47.703 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 199, 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "RepRequest": { 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "reqid": "client.4871.0:14132", 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "6.0", 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 101, 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "min_epoch": 100, 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "oid": "6:03799da1:::benchmark_data_smithi103_40346_object3352:head", 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "2" 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.704 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 200, 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 50, 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.019718+0000" 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.019734+0000" 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.020013+0000" 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.024858+0000" 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.029910+0000" 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.736652+0000" 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.031515+0000" 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.709 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.032301+0000" 2024-09-15T12:31:47.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.032719+0000" 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.714 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.715 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 53, 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.036256+0000" 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.036274+0000" 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.717 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.039675+0000" 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.718 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.042338+0000" 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.043685+0000" 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.719 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.736756+0000" 2024-09-15T12:31:47.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.044487+0000" 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.721 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.045161+0000" 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.727 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "peering_event", 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 158, 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "PeeringEvent": { 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "from": "1", 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "pgid": "6.b", 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "sent": 101, 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "requested": 100, 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "evt": "epoch_sent: 101 epoch_requested: 100 MLogRec from 1 log log((92'485,98'878], crt=98'878) pi ([0,0] all_participants={} intervals=[]) pg_lease(ru 0.000000000s ub 260.135070801s int 16.000000000s) +create_info", 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "events": [ 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.171784+0000" 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.171794+0000" 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.180452+0000" 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.729 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.180818+0000" 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.254865+0000" 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.731 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::AwaitMap", 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.398160+0000" 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.732 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGPeeringPipeline::Process", 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:47.522232+0000" 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 41, 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.000469+0000" 2024-09-15T12:31:47.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.000501+0000" 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.001878+0000" 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.735 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.003265+0000" 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.004480+0000" 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.764261+0000" 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.005187+0000" 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.005583+0000" 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.006014+0000" 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.742 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.744 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 46, 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.012163+0000" 2024-09-15T12:31:47.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.012179+0000" 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.012849+0000" 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.014957+0000" 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.747 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.020571+0000" 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.748 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.768799+0000" 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.021556+0000" 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.021772+0000" 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.750 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.021872+0000" 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.751 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.752 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.755 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 58, 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.048469+0000" 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.756 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.048486+0000" 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.050639+0000" 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.757 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.050994+0000" 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.758 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.053114+0000" 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.759 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.768963+0000" 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.053577+0000" 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.053999+0000" 2024-09-15T12:31:47.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.762 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 61, 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.057259+0000" 2024-09-15T12:31:47.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.057275+0000" 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.058124+0000" 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.059647+0000" 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.769 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.064893+0000" 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.770 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.769106+0000" 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.065536+0000" 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.772 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.773 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.774 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.777 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 65, 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.065886+0000" 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.065896+0000" 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.066345+0000" 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.067838+0000" 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.070116+0000" 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.786415+0000" 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.782422+0000" 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.783 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.788 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "client_request", 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 69, 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": { 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.076220+0000" 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.789 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.076238+0000" 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.078463+0000" 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.079247+0000" 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.791 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.080991+0000" 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:31:47.792 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "2024-09-15T12:31:45.796881+0000" 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.793 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "blocking_event": { 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "time_event": { 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "initiated_at": "0.000000" 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "background_recovery_sub", 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 200, 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: "detail": {} 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-15T12:31:47.800 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-15T12:31:47.801 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-15T12:31:49.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:49.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:49.995+0000 7f703a46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:50.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:50.003+0000 7f703a46c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:52.278 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:31:52.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:52.444+0000 7faa27f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:52.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:52.451+0000 7faa27f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:52.802 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2024-09-15T12:31:52.802 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:31:52.803 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3] 2024-09-15T12:31:52.803 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:31:52.803 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:31:54.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:31:54.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:54.824+0000 7f0680ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:54.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:31:54.830+0000 7f0680ebd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:31:56.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-15T12:31:58.903 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:32:01.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-15T12:32:03.904 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:32:03.904 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:32:03.905 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:32:03.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:32:04.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:04.017+0000 7fe308711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:04.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:04.027+0000 7fe308711640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:04.044 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:04.045 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:04.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:04.049 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:04.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:32:04.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:04.157+0000 7f30940fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:04.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:04.168+0000 7f30940fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:24.177 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:24.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:24.182 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:24.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:32:24.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:24.329+0000 7f9be2b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:24.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:24.340+0000 7f9be2b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:24.379 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:24.379 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:24.380 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:24.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:24.384 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:24.384 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:32:24.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:24.503+0000 7feb79951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:24.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:24.514+0000 7feb79951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:24.716 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:24.717 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:24.720 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:24.721 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:24.841 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:25.109 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:25.110 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:27.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:27.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:27.246+0000 7f377bd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:27.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:27.253+0000 7f377bd11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:27.317 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:27.595 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:27.596 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:29.597 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:29.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:29.706+0000 7fc486474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:29.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:29.713+0000 7fc486474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:29.722 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:32:29.722 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:32:29.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:32:29.794 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:29.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:29.847+0000 7f4e54391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:29.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:29.857+0000 7f4e54391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:30.046 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:30.046 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:30.142 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:32:30.142 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:32:30.160 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-15T12:32:28.142753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424280651,"num_objects":6469,"num_object_clones":0,"num_object_copies":19407,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":15151,"num_write_kb":690261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":55508992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15139,"ondisk_log_size":15139,"up":70,"acting":64,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":284692,"kb_used_data":84116,"kb_used_omap":0,"kb_used_meta":200512,"kb_avail":377202668,"statfs":{"total":386547056640,"available":386255532032,"internally_reserved":0,"allocated":86134784,"data_stored":1320420230,"data_compressed":62637784,"data_compressed_allocated":82448384,"data_compressed_original":1318928384,"omap_allocated":0,"internal_metadata":205324288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15859712,"num_objects":-242,"num_object_clones":0,"num_object_copies":-726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":891,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":242,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":804,"num_bytes_recovered":24969216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001282"},"pg_stats":[{"pgid":"6.f","version":"114'928","reported_seq":1016,"reported_epoch":114,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:32:26.346212+0000","last_change":"2024-09-15T12:32:22.364452+0000","last_active":"2024-09-15T12:32:26.346212+0000","last_peered":"2024-09-15T12:32:26.346212+0000","last_clean":"2024-09-15T12:32:16.223463+0000","last_became_active":"2024-09-15T12:32:22.364452+0000","last_became_peered":"2024-09-15T12:32:22.364452+0000","last_unstale":"2024-09-15T12:32:26.346212+0000","last_undegraded":"2024-09-15T12:32:22.251936+0000","last_fullsized":"2024-09-15T12:32:22.250619+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24707072,"num_objects":377,"num_object_clones":0,"num_object_copies":1131,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":927,"num_write_kb":41728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":377}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"114'970","reported_seq":1046,"reported_epoch":114,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:32:26.352123+0000","last_change":"2024-09-15T12:32:22.312963+0000","last_active":"2024-09-15T12:32:26.352123+0000","last_peered":"2024-09-15T12:32:26.352123+0000","last_clean":"2024-09-15T12:31:52.622812+0000","last_became_active":"2024-09-15T12:32:22.312963+0000","last_became_peered":"2024-09-15T12:32:22.312963+0000","last_unstale":"2024-09-15T12:32:26.352123+0000","last_undegraded":"2024-09-15T12:32:22.240799+0000","last_fullsized":"2024-09-15T12:32:22.239710+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":373,"num_object_clones":0,"num_object_copies":1119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":42944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":373}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"102'911","reported_seq":1045,"reported_epoch":102,"state":"stale+active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T12:31:50.561113+0000","last_change":"2024-09-15T12:31:47.482557+0000","last_active":"2024-09-15T12:31:50.561113+0000","last_peered":"2024-09-15T12:31:50.561113+0000","last_clean":"2024-09-15T12:30:27.840085+0000","last_became_active":"2024-09-15T12:31:47.362216+0000","last_became_peered":"2024-09-15T12:31:47.362216+0000","last_unstale":"2024-09-15T12:32:21.213841+0000","last_undegraded":"2024-09-15T12:31:46.772791+0000","last_fullsized":"2024-09-15T12:31:46.739995+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30015512,"num_objects":459,"num_object_clones":0,"num_object_copies":1377,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":911,"num_write_kb":43777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2555904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":294}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"102'971","reported_seq":1028,"reported_epoch":102,"state":"stale+active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:31:50.423381+0000","last_change":"2024-09-15T12:31:47.540032+0000","last_active":"2024-09-15T12:31:50.423381+0000","last_peered":"2024-09-15T12:31:50.423381+0000","last_clean":"2024-09-15T12:30:27.803630+0000","last_became_active":"2024-09-15T12:31:47.522270+0000","last_became_peered":"2024-09-15T12:31:47.522270+0000","last_unstale":"2024-09-15T12:32:21.213843+0000","last_undegraded":"2024-09-15T12:31:46.962189+0000","last_fullsized":"2024-09-15T12:31:46.925362+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":971,"log_dups_size":0,"ondisk_log_size":971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":454,"num_object_clones":0,"num_object_copies":1362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":970,"num_write_kb":45568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":269}],"blocked_by":[],"up_primary":3,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"114'981","reported_seq":1093,"reported_epoch":114,"state":"active+recovering+undersized+degraded","last_fresh":"2024-09-15T12:32:26.266962+0000","last_change":"2024-09-15T12:32:23.418584+0000","last_active":"2024-09-15T12:32:26.266962+0000","last_peered":"2024-09-15T12:32:26.266962+0000","last_clean":"2024-09-15T12:30:27.899645+0000","last_became_active":"2024-09-15T12:32:23.316099+0000","last_became_peered":"2024-09-15T12:32:23.316099+0000","last_unstale":"2024-09-15T12:32:26.266962+0000","last_undegraded":"2024-09-15T12:32:23.252558+0000","last_fullsized":"2024-09-15T12:32:23.248005+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24772608,"num_objects":378,"num_object_clones":0,"num_object_copies":1134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":980,"num_write_kb":43456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1572864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":170},{"shards":"2,3","objects":208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"114'944","reported_seq":1032,"reported_epoch":114,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:32:26.194851+0000","last_change":"2024-09-15T12:32:22.341233+0000","last_active":"2024-09-15T12:32:26.194851+0000","last_peered":"2024-09-15T12:32:26.194851+0000","last_clean":"2024-09-15T12:32:16.222367+0000","last_became_active":"2024-09-15T12:32:22.341233+0000","last_became_peered":"2024-09-15T12:32:22.341233+0000","last_unstale":"2024-09-15T12:32:26.194851+0000","last_undegraded":"2024-09-15T12:32:22.246931+0000","last_fullsized":"2024-09-15T12:32:22.245225+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":943,"num_write_kb":42432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"102'886","reported_seq":1021,"reported_epoch":102,"state":"stale+active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T12:31:50.568505+0000","last_change":"2024-09-15T12:31:47.817908+0000","last_active":"2024-09-15T12:31:50.568505+0000","last_peered":"2024-09-15T12:31:50.568505+0000","last_clean":"2024-09-15T12:30:27.729444+0000","last_became_active":"2024-09-15T12:31:47.689820+0000","last_became_peered":"2024-09-15T12:31:47.689820+0000","last_unstale":"2024-09-15T12:32:21.213846+0000","last_undegraded":"2024-09-15T12:31:46.919396+0000","last_fullsized":"2024-09-15T12:31:46.779174+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":415,"num_object_clones":0,"num_object_copies":1245,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":885,"num_write_kb":41600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[1,2],"avail_no_missing":["1","2"],"object_location_counts":[{"shards":"1,2","objects":278}],"blocked_by":[],"up_primary":3,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"102'937","reported_seq":1804,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T12:32:10.751197+0000","last_change":"2024-09-15T12:31:58.777958+0000","last_active":"2024-09-15T12:32:10.751197+0000","last_peered":"2024-09-15T12:32:10.751197+0000","last_clean":"2024-09-15T12:32:10.751197+0000","last_became_active":"2024-09-15T12:31:58.772002+0000","last_became_peered":"2024-09-15T12:31:58.772002+0000","last_unstale":"2024-09-15T12:32:10.751197+0000","last_undegraded":"2024-09-15T12:32:10.751197+0000","last_fullsized":"2024-09-15T12:32:10.751197+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":937,"log_dups_size":0,"ondisk_log_size":937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804224,"num_objects":409,"num_object_clones":0,"num_object_copies":1227,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":937,"num_write_kb":43072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":12648448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"102'954","reported_seq":1825,"reported_epoch":108,"state":"peering","last_fresh":"2024-09-15T12:32:09.765598+0000","last_change":"2024-09-15T12:32:09.765598+0000","last_active":"2024-09-15T12:32:09.765598+0000","last_peered":"2024-09-15T12:32:08.914975+0000","last_clean":"2024-09-15T12:30:27.980820+0000","last_became_active":"2024-09-15T12:31:47.460930+0000","last_became_peered":"2024-09-15T12:31:47.460930+0000","last_unstale":"2024-09-15T12:32:09.765598+0000","last_undegraded":"2024-09-15T12:32:09.765598+0000","last_fullsized":"2024-09-15T12:32:09.765598+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":390,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":954,"num_write_kb":43008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":12058624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":215,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-15T12:32:15.206817+0000","last_change":"2024-09-15T12:31:45.943096+0000","last_active":"2024-09-15T12:32:15.206817+0000","last_peered":"2024-09-15T12:32:15.206817+0000","last_clean":"2024-09-15T12:32:15.206817+0000","last_became_active":"2024-09-15T12:31:45.934244+0000","last_became_peered":"2024-09-15T12:31:45.934244+0000","last_unstale":"2024-09-15T12:32:15.206817+0000","last_undegraded":"2024-09-15T12:32:15.206817+0000","last_fullsized":"2024-09-15T12:32:15.206817+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":238,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-15T12:31:48.779249+0000","last_change":"2024-09-15T12:31:28.750646+0000","last_active":"2024-09-15T12:31:48.779249+0000","last_peered":"2024-09-15T12:31:48.779249+0000","last_clean":"2024-09-15T12:31:48.779249+0000","last_became_active":"2024-09-15T12:31:27.950462+0000","last_became_peered":"2024-09-15T12:31:27.950462+0000","last_unstale":"2024-09-15T12:32:21.213848+0000","last_undegraded":"2024-09-15T12:31:48.779249+0000","last_fullsized":"2024-09-15T12:31:48.779249+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.4","version":"102'969","reported_seq":1038,"reported_epoch":108,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:32:10.755068+0000","last_change":"2024-09-15T12:31:47.701063+0000","last_active":"2024-09-15T12:32:10.755068+0000","last_peered":"2024-09-15T12:32:10.755068+0000","last_clean":"2024-09-15T12:30:27.672452+0000","last_became_active":"2024-09-15T12:31:47.690089+0000","last_became_peered":"2024-09-15T12:31:47.690089+0000","last_unstale":"2024-09-15T12:32:10.755068+0000","last_undegraded":"2024-09-15T12:31:46.815460+0000","last_fullsized":"2024-09-15T12:31:46.783567+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28508160,"num_objects":435,"num_object_clones":0,"num_object_copies":1305,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":396,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":967,"num_write_kb":44864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":246}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":292,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T12:32:10.756273+0000","last_change":"2024-09-15T12:31:28.747451+0000","last_active":"2024-09-15T12:32:10.756273+0000","last_peered":"2024-09-15T12:32:10.756273+0000","last_clean":"2024-09-15T12:32:10.756273+0000","last_became_active":"2024-09-15T12:31:27.904819+0000","last_became_peered":"2024-09-15T12:31:27.904819+0000","last_unstale":"2024-09-15T12:32:10.756273+0000","last_undegraded":"2024-09-15T12:32:10.756273+0000","last_fullsized":"2024-09-15T12:32:10.756273+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"114'963","reported_seq":1793,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-15T12:32:25.996185+0000","last_change":"2024-09-15T12:32:16.248237+0000","last_active":"2024-09-15T12:32:25.996185+0000","last_peered":"2024-09-15T12:32:25.996185+0000","last_clean":"2024-09-15T12:32:25.996185+0000","last_became_active":"2024-09-15T12:32:16.243123+0000","last_became_peered":"2024-09-15T12:32:16.243123+0000","last_unstale":"2024-09-15T12:32:25.996185+0000","last_undegraded":"2024-09-15T12:32:25.996185+0000","last_fullsized":"2024-09-15T12:32:25.996185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26476544,"num_objects":404,"num_object_clones":0,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":962,"num_write_kb":43712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":11468800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":230,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-15T12:31:48.751991+0000","last_change":"2024-09-15T12:31:46.927093+0000","last_active":"2024-09-15T12:31:48.751991+0000","last_peered":"2024-09-15T12:31:48.751991+0000","last_clean":"2024-09-15T12:31:48.751991+0000","last_became_active":"2024-09-15T12:31:45.821667+0000","last_became_peered":"2024-09-15T12:31:45.821667+0000","last_unstale":"2024-09-15T12:32:21.213852+0000","last_undegraded":"2024-09-15T12:31:48.751991+0000","last_fullsized":"2024-09-15T12:31:48.751991+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"102'924","reported_seq":988,"reported_epoch":108,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:32:10.757330+0000","last_change":"2024-09-15T12:31:47.682525+0000","last_active":"2024-09-15T12:32:10.757330+0000","last_peered":"2024-09-15T12:32:10.757330+0000","last_clean":"2024-09-15T12:30:27.634611+0000","last_became_active":"2024-09-15T12:31:47.668861+0000","last_became_peered":"2024-09-15T12:31:47.668861+0000","last_unstale":"2024-09-15T12:32:10.757330+0000","last_undegraded":"2024-09-15T12:31:46.776457+0000","last_fullsized":"2024-09-15T12:31:46.743887+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25755648,"num_objects":393,"num_object_clones":0,"num_object_copies":1179,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":923,"num_write_kb":42112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":212}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":294,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-15T12:32:10.753401+0000","last_change":"2024-09-15T12:31:28.736383+0000","last_active":"2024-09-15T12:32:10.753401+0000","last_peered":"2024-09-15T12:32:10.753401+0000","last_clean":"2024-09-15T12:32:10.753401+0000","last_became_active":"2024-09-15T12:31:27.894575+0000","last_became_peered":"2024-09-15T12:31:27.894575+0000","last_unstale":"2024-09-15T12:32:10.753401+0000","last_undegraded":"2024-09-15T12:32:10.753401+0000","last_fullsized":"2024-09-15T12:32:10.753401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"114'994","reported_seq":1903,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-15T12:32:26.256223+0000","last_change":"2024-09-15T12:32:03.778355+0000","last_active":"2024-09-15T12:32:26.256223+0000","last_peered":"2024-09-15T12:32:26.256223+0000","last_clean":"2024-09-15T12:32:26.256223+0000","last_became_active":"2024-09-15T12:32:03.772806+0000","last_became_peered":"2024-09-15T12:32:03.772806+0000","last_unstale":"2024-09-15T12:32:26.256223+0000","last_undegraded":"2024-09-15T12:32:26.256223+0000","last_fullsized":"2024-09-15T12:32:26.256223+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":994,"log_dups_size":0,"ondisk_log_size":994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25624576,"num_objects":391,"num_object_clones":0,"num_object_copies":1173,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":993,"num_write_kb":44288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":11927552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":303,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-15T12:32:23.782278+0000","last_change":"2024-09-15T12:31:46.790636+0000","last_active":"2024-09-15T12:32:23.782278+0000","last_peered":"2024-09-15T12:32:23.782278+0000","last_clean":"2024-09-15T12:32:23.782278+0000","last_became_active":"2024-09-15T12:31:45.876303+0000","last_became_peered":"2024-09-15T12:31:45.876303+0000","last_unstale":"2024-09-15T12:32:23.782278+0000","last_undegraded":"2024-09-15T12:32:23.782278+0000","last_fullsized":"2024-09-15T12:32:23.782278+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1","version":"114'962","reported_seq":1050,"reported_epoch":114,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:32:26.037161+0000","last_change":"2024-09-15T12:32:22.330318+0000","last_active":"2024-09-15T12:32:26.037161+0000","last_peered":"2024-09-15T12:32:26.037161+0000","last_clean":"2024-09-15T12:32:16.223672+0000","last_became_active":"2024-09-15T12:32:22.330318+0000","last_became_peered":"2024-09-15T12:32:22.330318+0000","last_unstale":"2024-09-15T12:32:26.037161+0000","last_undegraded":"2024-09-15T12:32:22.246326+0000","last_fullsized":"2024-09-15T12:32:22.245233+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":962,"log_dups_size":0,"ondisk_log_size":962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25100288,"num_objects":383,"num_object_clones":0,"num_object_copies":1149,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":961,"num_write_kb":43008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-15T12:32:15.216470+0000","last_change":"2024-09-15T12:31:46.752388+0000","last_active":"2024-09-15T12:32:15.216470+0000","last_peered":"2024-09-15T12:32:15.216470+0000","last_clean":"2024-09-15T12:32:15.216470+0000","last_became_active":"2024-09-15T12:31:45.983474+0000","last_became_peered":"2024-09-15T12:31:45.983474+0000","last_unstale":"2024-09-15T12:32:15.216470+0000","last_undegraded":"2024-09-15T12:32:15.216470+0000","last_fullsized":"2024-09-15T12:32:15.216470+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"102'852","reported_seq":916,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-15T12:31:50.336985+0000","last_change":"2024-09-15T12:31:28.088305+0000","last_active":"2024-09-15T12:31:50.336985+0000","last_peered":"2024-09-15T12:31:50.336985+0000","last_clean":"2024-09-15T12:31:50.336985+0000","last_became_active":"2024-09-15T12:31:27.958296+0000","last_became_peered":"2024-09-15T12:31:27.958296+0000","last_unstale":"2024-09-15T12:32:21.213854+0000","last_undegraded":"2024-09-15T12:31:50.336985+0000","last_fullsized":"2024-09-15T12:31:50.336985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25690112,"num_objects":392,"num_object_clones":0,"num_object_copies":1176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":852,"num_write_kb":39808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":303,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-15T12:32:23.777133+0000","last_change":"2024-09-15T12:31:46.790166+0000","last_active":"2024-09-15T12:32:23.777133+0000","last_peered":"2024-09-15T12:32:23.777133+0000","last_clean":"2024-09-15T12:32:23.777133+0000","last_became_active":"2024-09-15T12:31:45.853137+0000","last_became_peered":"2024-09-15T12:31:45.853137+0000","last_unstale":"2024-09-15T12:32:23.777133+0000","last_undegraded":"2024-09-15T12:32:23.777133+0000","last_fullsized":"2024-09-15T12:32:23.777133+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"102'938","reported_seq":1017,"reported_epoch":108,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T12:32:10.759336+0000","last_change":"2024-09-15T12:31:47.804627+0000","last_active":"2024-09-15T12:32:10.759336+0000","last_peered":"2024-09-15T12:32:10.759336+0000","last_clean":"2024-09-15T12:30:27.971946+0000","last_became_active":"2024-09-15T12:31:47.415296+0000","last_became_peered":"2024-09-15T12:31:47.415296+0000","last_unstale":"2024-09-15T12:32:10.759336+0000","last_undegraded":"2024-09-15T12:31:46.849011+0000","last_fullsized":"2024-09-15T12:31:46.819434+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:29:48.698382+0000","last_clean_scrub_stamp":"2024-09-15T12:29:48.698382+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28180480,"num_objects":430,"num_object_clones":0,"num_object_copies":1290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":386,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":938,"num_write_kb":43776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[{"shards":"0,1","objects":242}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":238,"reported_epoch":102,"state":"stale+active+clean","last_fresh":"2024-09-15T12:31:48.745305+0000","last_change":"2024-09-15T12:31:28.742504+0000","last_active":"2024-09-15T12:31:48.745305+0000","last_peered":"2024-09-15T12:31:48.745305+0000","last_clean":"2024-09-15T12:31:48.745305+0000","last_became_active":"2024-09-15T12:31:27.886818+0000","last_became_peered":"2024-09-15T12:31:27.886818+0000","last_unstale":"2024-09-15T12:32:21.213856+0000","last_undegraded":"2024-09-15T12:31:48.745305+0000","last_fullsized":"2024-09-15T12:31:48.745305+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":16,"stat_sum":{"num_bytes":423690264,"num_objects":6466,"num_object_clones":0,"num_object_copies":19398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15072,"num_write_kb":689153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1691,"num_bytes_recovered":55508992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":77365248,"data_stored":1237712944,"data_compressed":58772455,"data_compressed_allocated":77357056,"data_compressed_original":1237712896,"omap_allocated":0,"internal_metadata":0},"log_size":15084,"ondisk_log_size":15084,"up":43,"acting":37,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762334,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54884,"kb_used_data":20308,"kb_used_omap":0,"kb_used_meta":34560,"kb_avail":94316956,"statfs":{"total":96636764160,"available":96580562944,"internally_reserved":0,"allocated":20795392,"data_stored":318275994,"data_compressed":15097888,"data_compressed_allocated":19873792,"data_compressed_original":317898752,"omap_allocated":0,"internal_metadata":35389440},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61580,"kb_used_data":17980,"kb_used_omap":0,"kb_used_meta":43584,"kb_avail":94310260,"statfs":{"total":96636764160,"available":96573706240,"internally_reserved":0,"allocated":18411520,"data_stored":279303061,"data_compressed":13246720,"data_compressed_allocated":17436672,"data_compressed_original":278904832,"omap_allocated":0,"internal_metadata":44630016},"hb_peers":[0,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":61,"seq":261993005295,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":74400,"kb_used_data":23568,"kb_used_omap":0,"kb_used_meta":50816,"kb_avail":94297440,"statfs":{"total":96636764160,"available":96560578560,"internally_reserved":0,"allocated":24133632,"data_stored":372714642,"data_compressed":17682384,"data_compressed_allocated":23273472,"data_compressed_original":372375552,"omap_allocated":0,"internal_metadata":52035584},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757697,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":93828,"kb_used_data":22260,"kb_used_omap":0,"kb_used_meta":71552,"kb_avail":94278012,"statfs":{"total":96636764160,"available":96540684288,"internally_reserved":0,"allocated":22794240,"data_stored":350126533,"data_compressed":16610792,"data_compressed_allocated":21864448,"data_compressed_original":349749248,"omap_allocated":0,"internal_metadata":73269248},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21827584,"data_stored":349175832,"data_compressed":16580736,"data_compressed_allocated":21823488,"data_compressed_original":349175808,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23003136,"data_stored":367984664,"data_compressed":17473880,"data_compressed_allocated":22999040,"data_compressed_original":367984640,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17158144,"data_stored":274530304,"data_compressed":13036168,"data_compressed_allocated":17158144,"data_compressed_original":274530304,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15376384,"data_stored":246022144,"data_compressed":11681671,"data_compressed_allocated":15376384,"data_compressed_original":246022144,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:32:30.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-backfill 6.e 6.d 6.c 6.4 6.3 2024-09-15T12:32:30.298 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:30.286+0000 7fdfa5b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:30.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:30.301+0000 7fdfa5b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:30.514 INFO:teuthology.orchestra.run.smithi103.stderr:instructing pg(s) [6.3,6.d] on osd.0 to force-backfill; instructing pg(s) [6.e] on osd.2 to force-backfill; instructing pg(s) [6.4,6.c] on osd.3 to force-backfill; 2024-09-15T12:32:30.514 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:32:32.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:32.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:32.162+0000 7f454fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:32.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:32.168+0000 7f454fd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:32.229 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:32.648 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:34.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:34.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:34.799+0000 7f562d263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:34.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:34.806+0000 7f562d263640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:34.841 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:35.265 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:35.265 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:35.535 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:32:35.535 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:32:35.535 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:32:35.535 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:32:35.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:32:35.538 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:32:35.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:32:35.647 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:32:35.650 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:35.650 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:32:37.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:37.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:37.407+0000 7f155c391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:37.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:37.413+0000 7f155c391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:37.458 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:37.868 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:37.869 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:39.870 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:40.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:40.014+0000 7f2c4e6bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:40.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:40.020+0000 7f2c4e6bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:40.066 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:40.481 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:40.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:40.652 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:32:40.781 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:32:40.785 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:40.785 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:32:42.482 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:42.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:42.630+0000 7f43f4d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:42.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:42.638+0000 7f43f4d51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:42.698 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:43.002 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:43.003 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:45.004 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:45.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:45.130+0000 7fe5ca191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:45.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:45.136+0000 7fe5ca191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:45.176 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:45.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:45.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:45.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:32:45.907 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:32:45.910 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:45.910 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:32:47.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:47.563+0000 7f6073591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:47.571+0000 7f6073591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:47.635 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:47.884 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:47.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:49.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:50.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:50.026+0000 7fec38391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:50.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:50.034+0000 7fec38391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:50.213 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:32:50.369 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:32:50.370 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:32:50.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:32:51.046 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:32:51.047 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:32:51.047 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:32:52.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:52.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:52.473+0000 7fc74484f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:52.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:52.478+0000 7fc74484f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:54.581 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:32:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:54.729+0000 7fdaf8e9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:54.737+0000 7fdaf8e9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.061 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:32:56.061 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:32:56.061 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:32:56.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:32:56.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.217+0000 7f3941991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.230+0000 7f3941991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.426 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:56.427 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:56.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:56.431 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:56.431 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:32:56.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.601+0000 7f0943751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.617+0000 7f0943751640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:56.792 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:56.793 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:56.795 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:56.796 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:56.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:32:56.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.938+0000 7f9baed51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:56.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:56.956+0000 7f9baed51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:57.018 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:57.019 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:57.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:57.022 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:57.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:32:57.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:57.169+0000 7f8784b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:57.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:57.185+0000 7f8784b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:32:57.217 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:32:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:32:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:32:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:32:57.218 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:32:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:32:57.222 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:32:57.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:32:57.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:57.553+0000 7f085c0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:57.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:32:57.568+0000 7f085c0fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:32:59.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:33:00.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:00.062+0000 7fc969191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:00.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:00.074+0000 7fc969191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:02.223 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:33:02.223 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:02.223 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 1] 2024-09-15T12:33:02.224 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:33:02.224 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:33:02.316 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:33:02.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:33:02.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:02.903+0000 7f3982991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:02.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:02.909+0000 7f3982991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:07.317 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:33:07.317 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:07.318 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:33:07.318 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:33:07.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:33:07.321 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:33:07.321 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:07.432 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:07.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:07.436 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:12.438 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:12.538 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:12.542 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:17.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:17.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:17.647 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:21.218 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:33:21.399 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:33:21.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:33:22.648 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:22.752 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:33:22.752 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:33:22.752 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:33:23.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:33:23.541 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:23.537+0000 7fcbd565c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:23.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:23.544+0000 7fcbd565c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:25.972 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:33:26.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:26.120+0000 7fe75e857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:26.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:26.128+0000 7fe75e857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:27.762 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:33:27.762 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:27.763 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 1] 2024-09-15T12:33:27.763 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:33:27.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:33:27.846 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:33:28.551 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:33:28.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:28.657+0000 7f534a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:28.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:28.663+0000 7f534a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:32.847 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:33:32.847 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:32.847 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:33:32.848 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:33:32.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:33:32.851 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:33:32.851 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:32.969 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:32.972 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:32.973 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:37.973 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:38.073 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:38.076 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:38.076 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:41.746 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:33:41.910 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:33:41.910 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:33:43.078 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:43.181 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:33:43.181 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:33:43.181 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:33:43.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:33:44.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:44.045+0000 7fd90f488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:44.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:44.051+0000 7fd90f488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:46.258 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:33:46.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:46.405+0000 7f3320cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:46.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:46.414+0000 7f3320cd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:48.190 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:33:48.190 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:48.191 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 2, 3, 1] 2024-09-15T12:33:48.191 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:33:48.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:33:48.275 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:33:48.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:33:48.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:48.807+0000 7f187f87f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:48.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:33:48.812+0000 7f187f87f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:33:53.276 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2024-09-15T12:33:53.276 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:33:53.277 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:33:53.277 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:33:53.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:33:53.280 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:33:53.280 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:53.391 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:53.394 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:53.394 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:33:58.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:33:58.492 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:33:58.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:33:58.495 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:34:02.232 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:34:02.381 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:34:02.382 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:34:03.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:34:03.626 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:34:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:34:03.627 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:34:04.383 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:04.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:04.512+0000 7fe049efd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:04.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:04.518+0000 7fe049efd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:05.070 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Removed 10777 objects 2024-09-15T12:34:05.070 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Clean up completed and total clean up time :156.218 2024-09-15T12:34:06.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:06.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:06.889+0000 7feafbf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:06.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:06.896+0000 7feafbf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:07.007 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-15T12:34:07.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-15T12:34:07.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:07.120+0000 7f2b09f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:07.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:07.135+0000 7f2b09f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.229 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_3' removed 2024-09-15T12:34:08.251 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-15T12:34:08.253 INFO:tasks.radosbench:Beginning radosbench... 2024-09-15T12:34:08.253 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-15T12:34:08.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_4 16 2024-09-15T12:34:08.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.407+0000 7f47ec191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.421+0000 7f47ec191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.637 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:34:08.637 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:08.637 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:34:08.637 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:34:08.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.774+0000 7f72df391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.785+0000 7f72df391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:08.820 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:08.821 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:08.824 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:08.824 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:08.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:34:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.966+0000 7f036226c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:08.978+0000 7f036226c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.126 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:09.127 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:09.128 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:09.130 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:09.131 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:09.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:34:09.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:09.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.245+0000 7f3264951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.256+0000 7f3264951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.280 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:09.280 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:09.280 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:09.280 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:09.281 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:09.282 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:09.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:09.284 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:09.284 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:34:09.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.310+0000 7f86bef51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.316+0000 7f86bef51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.410+0000 7f18a6ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.419+0000 7f18a6ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:09.441 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:09.442 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:09.442 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:09.442 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:09.442 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:09.442 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:09.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:09.445 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:09.750 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_4' created 2024-09-15T12:34:09.769 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-15T12:34:09.904 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.890+0000 7f78cc89c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:09.909 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:09.906+0000 7f78cc89c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:10.754 INFO:teuthology.orchestra.run.smithi103.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-15T12:34:11.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:11.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:11.598+0000 7f7e2c311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:11.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:11.603+0000 7f7e2c311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:11.775 DEBUG:teuthology.orchestra.run.smithi103:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-15T12:34:11.798 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-15T12:34:11.798 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-15T12:34:11.863 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:hints = 1 2024-09-15T12:34:11.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-15T12:34:11.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object prefix: benchmark_data_smithi103_46160 2024-09-15T12:34:11.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:34:11.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 0 0 0 0 0 0 - 0 2024-09-15T12:34:12.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 1 16 200 184 11.4993 11.5 0.093425 0.0839796 2024-09-15T12:34:13.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:13.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:13.849+0000 7fa0b0391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:13.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:13.855+0000 7fa0b0391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:13.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 2 16 404 388 12.1243 12.75 0.0891361 0.0803677 2024-09-15T12:34:14.445 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:34:14.446 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:14.446 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:34:14.446 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:34:14.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.576+0000 7f6018b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:14.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.588+0000 7f6018b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:14.611 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:14.612 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:14.612 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:14.612 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:14.612 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:14.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:14.617 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:14.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:34:14.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.740+0000 7fe391ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:14.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.758+0000 7fe391ed1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:14.854 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:14.855 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:14.858 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:14.858 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:14.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:34:14.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 3 16 588 572 11.9159 11.5 0.173873 0.0817574 2024-09-15T12:34:14.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.979+0000 7f478e151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:14.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:14.991+0000 7f478e151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:15.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:15.044 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:15.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:15.048 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:15.049 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:34:15.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:15.230+0000 7f7ae00fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:15.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:15.244+0000 7f7ae00fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:15.275 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:15.276 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:15.279 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:15.280 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:15.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 4 16 788 772 12.0616 12.5 0.0508869 0.0822306 2024-09-15T12:34:16.135 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:16.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:16.277+0000 7f932d8b4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:16.290 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:16.288+0000 7f932d8b4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:16.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 5 15 992 977 12.2116 12.8125 0.0645054 0.0812953 2024-09-15T12:34:17.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 6 16 1202 1186 12.3532 13.0625 0.121304 0.0802559 2024-09-15T12:34:18.533 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:18.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:18.669+0000 7f608ef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:18.680 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:18.678+0000 7f608ef91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:18.864 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 7 16 1403 1387 12.383 12.5625 0.103978 0.0800698 2024-09-15T12:34:19.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 8 16 1594 1578 12.3272 11.9375 0.0729563 0.0806585 2024-09-15T12:34:20.281 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 1] 2024-09-15T12:34:20.281 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:20.281 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [0, 2, 3, 1] 2024-09-15T12:34:20.282 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T12:34:20.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:34:20.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 9 16 1696 1680 11.6657 6.375 0.0741803 0.0804714 2024-09-15T12:34:20.961 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:21.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T12:34:21.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:21.069+0000 7fdaabf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:21.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:21.074+0000 7fdaabf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:21.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 10 16 1696 1680 10.499 0 - 0.0804714 2024-09-15T12:34:22.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 11 16 1696 1680 9.54455 0 - 0.0804714 2024-09-15T12:34:23.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 12 16 1696 1680 8.74915 0 - 0.0804714 2024-09-15T12:34:24.865 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 13 16 1696 1680 8.0761 0 - 0.0804714 2024-09-15T12:34:25.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 14 16 1696 1680 7.49921 0 - 0.0804714 2024-09-15T12:34:26.383 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T12:34:26.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~6s 2024-09-15T12:34:26.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 15 16 1696 1680 6.99923 0 - 0.0804714 2024-09-15T12:34:27.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 16 16 1696 1680 6.56177 0 - 0.0804714 2024-09-15T12:34:28.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 17 16 1696 1680 6.17577 0 - 0.0804714 2024-09-15T12:34:29.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 18 16 1696 1680 5.83266 0 - 0.0804714 2024-09-15T12:34:30.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 19 16 1696 1680 5.52567 0 - 0.0804714 2024-09-15T12:34:31.384 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 3, 1] 2024-09-15T12:34:31.384 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:31.384 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-15T12:34:31.384 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T12:34:31.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-15T12:34:31.388 INFO:tasks.ceph.osd.0:Started 2024-09-15T12:34:31.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:34:31.500 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:34:31.504 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:31.504 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:34:31.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:34:31.865726+0000 min lat: 0.0149245 max lat: 0.174965 avg lat: 0.0804714 2024-09-15T12:34:31.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:34:31.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 20 16 1696 1680 5.24939 0 - 0.0804714 2024-09-15T12:34:32.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-15T12:34:32.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 21 16 1696 1680 4.99942 0 - 0.0804714 2024-09-15T12:34:33.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 22 16 1696 1680 4.77218 0 - 0.0804714 2024-09-15T12:34:34.866 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 23 16 1696 1680 4.5647 0 - 0.0804714 2024-09-15T12:34:35.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 24 16 1696 1680 4.3745 0 - 0.0804714 2024-09-15T12:34:36.505 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:34:36.608 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:34:36.610 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:36.611 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:34:36.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 25 16 1696 1680 4.19953 0 - 0.0804714 2024-09-15T12:34:37.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 26 16 1696 1680 4.03801 0 - 0.0804714 2024-09-15T12:34:38.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 27 16 1696 1680 3.88845 0 - 0.0804714 2024-09-15T12:34:39.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 28 16 1696 1680 3.74958 0 - 0.0804714 2024-09-15T12:34:40.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 29 16 1696 1680 3.62029 0 - 0.0804714 2024-09-15T12:34:41.235 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:34:41.455 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:34:41.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:34:41.612 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:34:41.715 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:34:41.715 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [] 2024-09-15T12:34:41.715 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:34:41.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 30 16 1696 1680 3.49962 0 - 0.0804714 2024-09-15T12:34:42.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 31 16 1696 1680 3.38673 0 - 0.0804714 2024-09-15T12:34:43.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:43.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:43.585+0000 7f869d65c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:43.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:43.592+0000 7f869d65c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:43.867 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 32 16 1758 1742 3.40197 0.168478 0.0254234 0.293532 2024-09-15T12:34:44.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 33 16 1963 1947 3.6871 12.8125 0.115903 0.271022 2024-09-15T12:34:45.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 34 15 2170 2155 3.96097 13 0.0629114 0.252228 2024-09-15T12:34:46.005 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:46.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:46.148+0000 7fcdbb874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:46.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:46.156+0000 7fcdbb874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:46.725 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 1, 0] 2024-09-15T12:34:46.725 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:46.725 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:34:46.726 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:34:46.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:46.854+0000 7f1714493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:46.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 35 16 2365 2349 4.19419 12.125 0.107397 0.238063 2024-09-15T12:34:46.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:46.867+0000 7f1714493640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:46.914 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:46.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:46.919 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:46.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:34:47.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.060+0000 7fc2f6d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.075+0000 7fc2f6d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.228 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:47.228 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:47.228 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:47.228 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:47.229 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:47.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:47.233 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:47.234 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:34:47.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.379+0000 7f4450151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.394+0000 7f4450151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:47.416 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:47.417 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:47.419 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:47.420 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:47.420 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:34:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.550+0000 7f9aa1191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:47.564+0000 7f9aa1191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:47.627 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:47.628 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:47.631 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:47.632 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:47.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 36 16 2558 2542 4.41272 12.0625 0.0509708 0.226339 2024-09-15T12:34:48.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:48.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:48.648+0000 7f176dd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:48.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:48.656+0000 7f176dd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:48.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 37 16 2761 2745 4.63632 12.6875 0.0488176 0.215473 2024-09-15T12:34:49.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 38 16 2970 2954 4.85803 13.0625 0.118786 0.205614 2024-09-15T12:34:50.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 39 16 3179 3163 5.06837 13.0625 0.0580674 0.197044 2024-09-15T12:34:50.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:51.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:51.099+0000 7f15fc391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:51.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:51.109+0000 7f15fc391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:51.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:34:51.867679+0000 min lat: 0.0149245 max lat: 23.3782 avg lat: 0.191428 2024-09-15T12:34:51.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:34:51.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 40 16 3357 3341 5.21976 11.125 0.106071 0.191428 2024-09-15T12:34:52.633 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 1, 0] 2024-09-15T12:34:52.633 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:52.633 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:34:52.633 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:34:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:52.791+0000 7f8cecf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:52.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:52.801+0000 7f8cecf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:52.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 41 16 3552 3536 5.38967 12.1875 0.0396766 0.185329 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:52.877 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:52.878 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:52.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:52.881 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:52.882 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:34:53.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.023+0000 7f9c1dd51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.036+0000 7f9c1dd51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:53.161 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:53.162 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:53.165 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:53.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:34:53.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.306+0000 7f8714991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.321+0000 7f8714991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.344 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:53.344 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:53.345 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:53.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:53.349 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:53.349 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:34:53.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.478+0000 7fa15d8c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.496 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:53.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.495+0000 7fa15d8c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:34:53.570 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:34:53.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:34:53.574 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:34:53.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.637+0000 7fd5636fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:53.646+0000 7fd5636fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:53.868 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 42 16 3739 3723 5.53959 11.6875 0.089947 0.180261 2024-09-15T12:34:54.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 43 16 3940 3924 5.70288 12.5625 0.0594441 0.175135 2024-09-15T12:34:55.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 44 16 4136 4120 5.85166 12.25 0.0486754 0.170753 2024-09-15T12:34:56.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:34:56.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:56.176+0000 7fa258ad1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:56.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:56.184+0000 7fa258ad1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:56.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 45 16 4340 4324 6.00493 12.75 0.0557262 0.166422 2024-09-15T12:34:57.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 46 16 4532 4516 6.13522 12 0.0360142 0.162708 2024-09-15T12:34:58.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:34:58.575 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 1, 0] 2024-09-15T12:34:58.575 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:34:58.575 INFO:tasks.thrashosds.thrasher:Killing osd 0, live_osds are [2, 3, 1, 0] 2024-09-15T12:34:58.575 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T12:34:58.575 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:34:58.695 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:58.691+0000 7f12a1391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:58.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:34:58.698+0000 7f12a1391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:34:58.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 47 16 4693 4677 6.21876 10.0625 0.0164886 0.160002 2024-09-15T12:34:59.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 48 16 4693 4677 6.0892 0 - 0.160002 2024-09-15T12:35:00.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T12:35:00.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 49 16 4693 4677 5.96493 0 - 0.160002 2024-09-15T12:35:01.869 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 50 16 4693 4677 5.84562 0 - 0.160002 2024-09-15T12:35:02.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 51 16 4693 4677 5.731 0 - 0.160002 2024-09-15T12:35:03.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 52 16 4693 4677 5.62078 0 - 0.160002 2024-09-15T12:35:04.676 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T12:35:04.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 53 16 4693 4677 5.51472 0 - 0.160002 2024-09-15T12:35:05.541 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-15T12:35:05.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 54 16 4693 4677 5.41259 0 - 0.160002 2024-09-15T12:35:06.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 55 16 4693 4677 5.31418 0 - 0.160002 2024-09-15T12:35:07.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 56 16 4693 4677 5.21928 0 - 0.160002 2024-09-15T12:35:08.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 57 16 4693 4677 5.12771 0 - 0.160002 2024-09-15T12:35:09.677 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 3, 1] 2024-09-15T12:35:09.677 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:35:09.678 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:35:09.678 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:35:09.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:09.794+0000 7fc88c8ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:09.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:09.803+0000 7fc88c8ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:09.870 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 58 16 4693 4677 5.0393 0 - 0.160002 2024-09-15T12:35:10.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 59 16 4693 4677 4.95389 0 - 0.160002 2024-09-15T12:35:11.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:35:11.870249+0000 min lat: 0.0149245 max lat: 23.3782 avg lat: 0.160002 2024-09-15T12:35:11.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:35:11.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 60 16 4693 4677 4.87132 0 - 0.160002 2024-09-15T12:35:12.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 61 16 4693 4677 4.79146 0 - 0.160002 2024-09-15T12:35:13.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 62 16 4693 4677 4.71417 0 - 0.160002 2024-09-15T12:35:14.871 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 63 16 4693 4677 4.63934 0 - 0.160002 2024-09-15T12:35:15.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 64 16 4693 4677 4.56685 0 - 0.160002 2024-09-15T12:35:16.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 65 16 4693 4677 4.49659 0 - 0.160002 2024-09-15T12:35:17.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 66 16 4693 4677 4.42846 0 - 0.160002 2024-09-15T12:35:18.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 67 16 4693 4677 4.36236 0 - 0.160002 2024-09-15T12:35:19.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 68 16 4693 4677 4.29821 0 - 0.160002 2024-09-15T12:35:20.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 69 16 4693 4677 4.23591 0 - 0.160002 2024-09-15T12:35:21.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 70 16 4693 4677 4.1754 0 - 0.160002 2024-09-15T12:35:22.872 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 71 16 4693 4677 4.11659 0 - 0.160002 2024-09-15T12:35:23.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 72 16 4693 4677 4.05941 0 - 0.160002 2024-09-15T12:35:24.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 73 16 4693 4677 4.0038 0 - 0.160002 2024-09-15T12:35:25.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 74 16 4693 4677 3.9497 0 - 0.160002 2024-09-15T12:35:26.770 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:35:26.770 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:35:26.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:35:26.775 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:35:26.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:35:26.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 75 16 4693 4677 3.89703 0 - 0.160002 2024-09-15T12:35:26.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:26.893+0000 7f1cd80d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:26.905 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:26.903+0000 7f1cd80d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:35:26.928 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:35:26.929 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:35:26.929 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:35:26.931 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:26.932 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:35:26.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:35:26.960 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:35:26.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:35:27.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:27.035+0000 7fc789591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:27.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:27.044+0000 7fc789591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:35:27.061 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:35:27.062 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:35:27.064 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:27.064 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:35:27.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:35:27.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:27.166+0000 7f7ad1111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:27.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:27.174+0000 7f7ad1111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:35:27.191 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:35:27.192 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:35:27.192 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:35:27.192 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:35:27.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:27.194 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:35:27.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 76 16 4695 4679 3.8474 0.00431034 29.1706 0.172425 2024-09-15T12:35:28.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 77 16 4898 4882 3.96219 12.6875 0.130332 0.252221 2024-09-15T12:35:28.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:35:29.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:29.081+0000 7eff0befd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:29.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:29.087+0000 7eff0befd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:29.094 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:35:29.345 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:35:29.346 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:35:29.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 78 16 5095 5079 4.06923 12.3125 0.0639117 0.245649 2024-09-15T12:35:30.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 79 16 5302 5286 4.18147 12.9375 0.0977453 0.239012 2024-09-15T12:35:31.347 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:35:31.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:31.463+0000 7f6b43151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:31.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:31.470+0000 7f6b43151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:31.478 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:35:31.746 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:35:31.746 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:35:31.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:35:31.872780+0000 min lat: 0.0149245 max lat: 29.3649 avg lat: 0.233213 2024-09-15T12:35:31.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:35:31.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 80 16 5502 5486 4.28544 12.5 0.055725 0.233213 2024-09-15T12:35:32.195 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0] live_osds: [2, 3, 1] 2024-09-15T12:35:32.196 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:35:32.196 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [2, 3, 1] 2024-09-15T12:35:32.196 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:35:32.196 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:35:32.268 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:35:32.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 81 16 5577 5561 4.29039 4.6875 0.0179522 0.231096 2024-09-15T12:35:33.747 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:35:33.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:33.861+0000 7f822be7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:33.869 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:35:33.866+0000 7f822be7d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:35:33.873 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 82 16 5577 5561 4.23807 0 - 0.231096 2024-09-15T12:35:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.0 2024-09-15T12:35:34.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 83 16 5577 5561 4.18701 0 - 0.231096 2024-09-15T12:35:35.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 84 16 5577 5561 4.13716 0 - 0.231096 2024-09-15T12:35:36.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 85 16 5577 5561 4.08849 0 - 0.231096 2024-09-15T12:35:37.269 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [0, 1] live_osds: [2, 3] 2024-09-15T12:35:37.269 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:35:37.269 INFO:tasks.thrashosds.thrasher:Reviving osd 0 2024-09-15T12:35:37.269 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T12:35:37.269 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-15T12:35:37.272 INFO:tasks.ceph.osd.0:Started 2024-09-15T12:35:37.273 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:35:37.383 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:35:37.386 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:37.387 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:35:37.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 86 16 5577 5561 4.04095 0 - 0.231096 2024-09-15T12:35:38.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 87 16 5577 5561 3.9945 0 - 0.231096 2024-09-15T12:35:39.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 88 16 5577 5561 3.94911 0 - 0.231096 2024-09-15T12:35:40.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 89 16 5577 5561 3.90474 0 - 0.231096 2024-09-15T12:35:41.874 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 90 16 5577 5561 3.86136 0 - 0.231096 2024-09-15T12:35:42.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:35:42.487 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:35:42.490 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:42.490 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T12:35:42.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 91 16 5577 5561 3.81892 0 - 0.231096 2024-09-15T12:35:43.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 92 16 5577 5561 3.77741 0 - 0.231096 2024-09-15T12:35:44.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 93 16 5577 5561 3.7368 0 - 0.231096 2024-09-15T12:35:45.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 94 16 5577 5561 3.69705 0 - 0.231096 2024-09-15T12:35:46.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 95 16 5577 5561 3.65813 0 - 0.231096 2024-09-15T12:35:47.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T12:35:47.596 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:35:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:35:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:35:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:35:47.596 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 15, 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.769370+0000" 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.769422+0000" 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.783270+0000" 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.783606+0000" 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.874174+0000" 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.875229+0000" 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.875688+0000" 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.875909+0000" 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.876007+0000", 2024-09-15T12:35:47.602 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "PGActivationBlocker", 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "7.1" 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:35:47.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 16, 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:35:47.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.770246+0000" 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.770273+0000" 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.783859+0000" 2024-09-15T12:35:47.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:35:47.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.875834+0000" 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.876699+0000" 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.877099+0000" 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.877291+0000", 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "CommonPGPipeline:::wait_for_active", 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1, 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 15 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.621 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 17, 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.770734+0000" 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.770750+0000" 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.876315+0000" 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.877385+0000" 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.624 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.878797+0000" 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:35:46.879213+0000", 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "blocker": { 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "op_type": "ClientRequest::PGPipeline::await_map", 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "waiting": 1, 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: "held_by_operation_id": 16 2024-09-15T12:35:47.626 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.627 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:35:47.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.630 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:35:47.634 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:35:47.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 96 16 5577 5561 3.62002 0 - 0.231096 2024-09-15T12:35:48.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 97 16 5577 5561 3.5827 0 - 0.231096 2024-09-15T12:35:49.875 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 98 16 5577 5561 3.54615 0 - 0.231096 2024-09-15T12:35:50.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 99 16 5577 5561 3.51033 0 - 0.231096 2024-09-15T12:35:51.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:2024-09-15T12:35:51.875025+0000 min lat: 0.0149245 max lat: 29.3649 avg lat: 0.231096 2024-09-15T12:35:51.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T12:35:51.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 100 16 5577 5561 3.47522 0 - 0.231096 2024-09-15T12:35:52.635 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [2, 3, 0] 2024-09-15T12:35:52.636 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:35:52.636 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-15T12:35:52.636 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T12:35:52.636 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-15T12:35:52.639 INFO:tasks.ceph.osd.1:Started 2024-09-15T12:35:52.639 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:35:52.750 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:35:52.753 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:52.754 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:35:52.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 101 16 5577 5561 3.44081 0 - 0.231096 2024-09-15T12:35:53.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 102 16 5577 5561 3.40708 0 - 0.231096 2024-09-15T12:35:54.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 103 16 5577 5561 3.374 0 - 0.231096 2024-09-15T12:35:55.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 104 16 5577 5561 3.34156 0 - 0.231096 2024-09-15T12:35:56.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 105 16 5577 5561 3.30973 0 - 0.231096 2024-09-15T12:35:57.754 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:35:57.854 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:35:57.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:35:57.857 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:35:57.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 106 16 5577 5561 3.27851 0 - 0.231096 2024-09-15T12:35:58.876 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 107 16 5577 5561 3.24787 0 - 0.231096 2024-09-15T12:35:59.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 108 16 5577 5561 3.2178 0 - 0.231096 2024-09-15T12:36:00.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 109 16 5577 5561 3.18828 0 - 0.231096 2024-09-15T12:36:01.768 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:36:01.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 110 16 5577 5561 3.15929 0 - 0.231096 2024-09-15T12:36:01.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:36:01.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:36:02.859 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:36:02.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout: 111 16 5577 5561 3.13083 0 - 0.231096 2024-09-15T12:36:02.970 INFO:teuthology.orchestra.run.smithi103.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T12:36:02.973 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:02.974 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T12:36:03.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total time run: 111.25 2024-09-15T12:36:03.877 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Total writes made: 5577 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Write size: 65536 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Object size: 65536 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Bandwidth (MB/sec): 3.13316 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Bandwidth: 5.31321 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max bandwidth (MB/sec): 13.0625 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min bandwidth (MB/sec): 0 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average IOPS: 50 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev IOPS: 85.0152 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max IOPS: 209 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min IOPS: 0 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Average Latency(s): 0.318911 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Stddev Latency(s): 2.57753 2024-09-15T12:36:03.878 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Max latency(s): 30.9356 2024-09-15T12:36:03.879 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Min latency(s): 0.0149245 2024-09-15T12:36:03.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:03.908 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T12:36:04.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:04.028+0000 7ff068497640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:04.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:04.034+0000 7ff068497640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:04.172 INFO:teuthology.orchestra.run.smithi103.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-15T12:36:04.313 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T12:36:04.314 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T12:36:06.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:06.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:06.480+0000 7f728e991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:06.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:06.491+0000 7f728e991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:07.975 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout: "ops_in_flight": [ 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 482, 2024-09-15T12:36:08.190 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.5217.0:6210", 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "7.0", 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 148, 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 147, 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "7:0082d764:::benchmark_data_smithi103_46160_object630:head", 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "2" 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "background_recovery_sub", 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 481, 2024-09-15T12:36:08.191 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": {} 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 470, 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.053541+0000" 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.053554+0000" 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.192 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.054139+0000" 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.193 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.054424+0000" 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.056146+0000" 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.057141+0000" 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.057832+0000" 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.198 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.058535+0000" 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.199 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.200 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.060833+0000" 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.201 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 472, 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.063488+0000" 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.203 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.063528+0000" 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.066451+0000" 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.066643+0000" 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.205 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.069505+0000" 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.070240+0000" 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.070573+0000" 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.209 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.071219+0000" 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.074102+0000" 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.212 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.213 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 478, 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.113354+0000" 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.113372+0000" 2024-09-15T12:36:08.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.114149+0000" 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.115552+0000" 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.119378+0000" 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.119852+0000" 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.120184+0000" 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.219 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.121097+0000" 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.126957+0000" 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.223 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "replicated_request", 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 479, 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "RepRequest": { 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "reqid": "client.5217.0:6207", 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "7.d", 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 148, 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min_epoch": 147, 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "oid": "7:be11b7a2:::benchmark_data_smithi103_46160_object627:head", 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: "from": "2" 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.225 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 392, 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.583223+0000" 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.583253+0000" 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.583496+0000" 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.584203+0000" 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.587290+0000" 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.644845+0000" 2024-09-15T12:36:08.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.587981+0000" 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.230 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.588302+0000" 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.232 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.589092+0000" 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.234 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.593745+0000" 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.644207+0000" 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.236 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 403, 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.649669+0000" 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.649686+0000" 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.237 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.650036+0000" 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.650302+0000" 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.653309+0000" 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.239 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.727541+0000" 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.240 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.653976+0000" 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.241 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.654309+0000" 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.242 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.243 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.654969+0000" 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.244 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.245 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.657656+0000" 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.246 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.725782+0000" 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 416, 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.722646+0000" 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.722673+0000" 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.248 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.722903+0000" 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.723033+0000" 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.723341+0000" 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.250 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.767527+0000" 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.724006+0000" 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.252 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.724270+0000" 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.254 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.724684+0000" 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.256 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.727812+0000" 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.257 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.766750+0000" 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 422, 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.753291+0000" 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.753312+0000" 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.753627+0000" 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.753788+0000" 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.754282+0000" 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.261 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.782235+0000" 2024-09-15T12:36:08.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.754960+0000" 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.263 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.755324+0000" 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.264 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.265 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.755983+0000" 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.266 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.267 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.762199+0000" 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.268 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.780324+0000" 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.269 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 428, 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.772177+0000" 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.772203+0000" 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.270 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.772454+0000" 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.271 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.772663+0000" 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773101+0000" 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.817947+0000" 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773787+0000" 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.274 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.774411+0000" 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.277 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.775087+0000" 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.784082+0000" 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.279 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.816997+0000" 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 430, 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.777042+0000" 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.777075+0000" 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.777431+0000" 2024-09-15T12:36:08.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.777635+0000" 2024-09-15T12:36:08.283 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.779728+0000" 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.827885+0000" 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.780940+0000" 2024-09-15T12:36:08.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.781429+0000" 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.287 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.288 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.783177+0000" 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.290 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.792074+0000" 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.826920+0000" 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.292 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 417, 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.727723+0000" 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.727737+0000" 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.727943+0000" 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.294 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.728051+0000" 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.733129+0000" 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.838272+0000" 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.296 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.733782+0000" 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.734111+0000" 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.298 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.734818+0000" 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.300 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.301 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.737523+0000" 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.837688+0000" 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.303 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 435, 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.833508+0000" 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.833547+0000" 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.304 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.833824+0000" 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.305 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.833991+0000" 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.834367+0000" 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.910066+0000" 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.835050+0000" 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.835362+0000" 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.309 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.836003+0000" 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.311 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.312 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.839386+0000" 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.909064+0000" 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.314 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 437, 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.838647+0000" 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.838659+0000" 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.315 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.838846+0000" 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.316 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.838948+0000" 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.844391+0000" 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.918797+0000" 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.318 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.845066+0000" 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.845470+0000" 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.320 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.846144+0000" 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.848840+0000" 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.917774+0000" 2024-09-15T12:36:08.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 439, 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.325 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.853858+0000" 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.853886+0000" 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.854123+0000" 2024-09-15T12:36:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.854313+0000" 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.855426+0000" 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.328 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.920501+0000" 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.856118+0000" 2024-09-15T12:36:08.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.856431+0000" 2024-09-15T12:36:08.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.331 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.857083+0000" 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.859831+0000" 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.919328+0000" 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.335 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 444, 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.336 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.884262+0000" 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.884270+0000" 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.884439+0000" 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.884535+0000" 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.884837+0000" 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.339 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.933191+0000" 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.885279+0000" 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.885521+0000" 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.342 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.885932+0000" 2024-09-15T12:36:08.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.889335+0000" 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.345 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.932261+0000" 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.346 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 446, 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886554+0000" 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.347 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886566+0000" 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886714+0000" 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886863+0000" 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.887944+0000" 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.935104+0000" 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.889016+0000" 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.351 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.889521+0000" 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.353 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.894466+0000" 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.354 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.355 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.896873+0000" 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.356 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.934176+0000" 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 449, 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.912518+0000" 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.358 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.912548+0000" 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.913253+0000" 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.914191+0000" 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.360 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.914710+0000" 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.361 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.995072+0000" 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.915150+0000" 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.915346+0000" 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.916013+0000" 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.365 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.366 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.921585+0000" 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.993127+0000" 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.368 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 453, 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.932434+0000" 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.932450+0000" 2024-09-15T12:36:08.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.932755+0000" 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.933187+0000" 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.371 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.935506+0000" 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.372 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.017027+0000" 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.936165+0000" 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.936483+0000" 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.375 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.937126+0000" 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.376 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.377 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.939583+0000" 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.378 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.008817+0000" 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.379 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 456, 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.943022+0000" 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.943051+0000" 2024-09-15T12:36:08.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.943421+0000" 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.381 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.943917+0000" 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.945794+0000" 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.383 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.019236+0000" 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.946558+0000" 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.946880+0000" 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.385 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.386 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.947609+0000" 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.950164+0000" 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.389 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.016881+0000" 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.390 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 458, 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.959726+0000" 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.391 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.959743+0000" 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.965145+0000" 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.965801+0000" 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.393 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.971260+0000" 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.057931+0000" 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.971969+0000" 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.396 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.972290+0000" 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.398 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.972943+0000" 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.975501+0000" 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.400 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.401 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.056608+0000" 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 461, 2024-09-15T12:36:08.402 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.986445+0000" 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.986459+0000" 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.403 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.986785+0000" 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.988628+0000" 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.991823+0000" 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.405 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.081534+0000" 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.993075+0000" 2024-09-15T12:36:08.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.407 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.993454+0000" 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.995236+0000" 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.410 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.000371+0000" 2024-09-15T12:36:08.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.412 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.080916+0000" 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 462, 2024-09-15T12:36:08.413 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.992076+0000" 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.992093+0000" 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.992384+0000" 2024-09-15T12:36:08.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.993339+0000" 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.995397+0000" 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.416 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.103574+0000" 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.997132+0000" 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.417 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.997470+0000" 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.419 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.998104+0000" 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.421 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.008426+0000" 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.422 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.102611+0000" 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.423 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 464, 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.997231+0000" 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.997243+0000" 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.997744+0000" 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.425 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.998024+0000" 2024-09-15T12:36:08.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.005009+0000" 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.109025+0000" 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.007345+0000" 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.010144+0000" 2024-09-15T12:36:08.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.430 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.016697+0000" 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.432 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.021119+0000" 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.108425+0000" 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 476, 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.096496+0000" 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.435 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.096533+0000" 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.098055+0000" 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.100235+0000" 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.437 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.101525+0000" 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.102149+0000" 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.102465+0000" 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.441 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.103139+0000" 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.443 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.106885+0000" 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.151807+0000" 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 480, 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.116454+0000" 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.446 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.116464+0000" 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.116967+0000" 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.447 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.117194+0000" 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.117614+0000" 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.449 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.118245+0000" 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.450 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.118580+0000" 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.452 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.119108+0000" 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.454 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.120900+0000" 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.456 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "background_recovery", 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 33554539, 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "pgid": "7.8", 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_detail": {} 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.457 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 370, 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.485202+0000" 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.485231+0000" 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.487388+0000" 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.459 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.487584+0000" 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.489354+0000" 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.552153+0000" 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.461 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.492981+0000" 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.462 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.493319+0000" 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.493979+0000" 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.466 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.498842+0000" 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.551608+0000" 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 373, 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.493736+0000" 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.493750+0000" 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.469 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.493961+0000" 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.470 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.494695+0000" 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.498752+0000" 2024-09-15T12:36:08.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.472 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.577742+0000" 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.501860+0000" 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.505384+0000" 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.475 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.507733+0000" 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.477 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.513640+0000" 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.575324+0000" 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.479 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 380, 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.537292+0000" 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.480 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.537314+0000" 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.538129+0000" 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.538339+0000" 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.539037+0000" 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.600398+0000" 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.539644+0000" 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.539986+0000" 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.485 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.486 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.540415+0000" 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.543707+0000" 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.489 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.599702+0000" 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 390, 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.573751+0000" 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.491 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.573766+0000" 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.575061+0000" 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.575258+0000" 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.493 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.576116+0000" 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.637046+0000" 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.577058+0000" 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.495 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.577659+0000" 2024-09-15T12:36:08.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.497 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.578490+0000" 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.499 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.584374+0000" 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.500 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.632119+0000" 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.501 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 394, 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.600468+0000" 2024-09-15T12:36:08.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.600480+0000" 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.600957+0000" 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.503 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.601418+0000" 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.607652+0000" 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.505 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.667094+0000" 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.608630+0000" 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.609508+0000" 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.507 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.508 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.509 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.613094+0000" 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.510 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.615321+0000" 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.511 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.665537+0000" 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.512 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 397, 2024-09-15T12:36:08.513 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.629534+0000" 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.629551+0000" 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.514 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.629724+0000" 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.515 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.629896+0000" 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.630984+0000" 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.692517+0000" 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.632078+0000" 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.517 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.518 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.632429+0000" 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.519 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.633023+0000" 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.521 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.639239+0000" 2024-09-15T12:36:08.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.690455+0000" 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.524 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 404, 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.657956+0000" 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.657968+0000" 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.658162+0000" 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.658290+0000" 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.659313+0000" 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.527 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.734023+0000" 2024-09-15T12:36:08.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.660217+0000" 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.660718+0000" 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.531 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.661475+0000" 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.532 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.533 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.668490+0000" 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.733005+0000" 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.535 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 407, 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.673503+0000" 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.673536+0000" 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.536 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.674154+0000" 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.674364+0000" 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.674982+0000" 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.746715+0000" 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.539 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.675526+0000" 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.675756+0000" 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.542 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.676148+0000" 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.544 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.678189+0000" 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.746066+0000" 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.546 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 410, 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.685223+0000" 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.685257+0000" 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.685471+0000" 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.685616+0000" 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.690839+0000" 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.550 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.760566+0000" 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.691725+0000" 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.693076+0000" 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.554 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.693742+0000" 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.702216+0000" 2024-09-15T12:36:08.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.748453+0000" 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 418, 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.744859+0000" 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.744876+0000" 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.559 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.745028+0000" 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.560 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.746904+0000" 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.747648+0000" 2024-09-15T12:36:08.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.562 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.873731+0000" 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.752139+0000" 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.563 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.757353+0000" 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.564 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.566 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.761638+0000" 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.567 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773097+0000" 2024-09-15T12:36:08.568 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.569 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.872879+0000" 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 419, 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.747568+0000" 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.747586+0000" 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.571 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.747893+0000" 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.748111+0000" 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.573 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.752423+0000" 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.574 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.877268+0000" 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.760045+0000" 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.760833+0000" 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.576 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.763636+0000" 2024-09-15T12:36:08.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.579 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.777850+0000" 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.580 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.874883+0000" 2024-09-15T12:36:08.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.582 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 420, 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.749652+0000" 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.749675+0000" 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.749889+0000" 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.750094+0000" 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.753079+0000" 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.585 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.881751+0000" 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.761300+0000" 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.764699+0000" 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.589 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.766605+0000" 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.590 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.770495+0000" 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.789352+0000" 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.880708+0000" 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.593 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 424, 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.759238+0000" 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.759270+0000" 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.595 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.759568+0000" 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.759708+0000" 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.761807+0000" 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.597 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883928+0000" 2024-09-15T12:36:08.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.765243+0000" 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.599 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.766947+0000" 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.600 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.601 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.768156+0000" 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.602 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773669+0000" 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.800781+0000" 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.604 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883046+0000" 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.605 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 429, 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.606 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773860+0000" 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.773870+0000" 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.774154+0000" 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.607 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.608 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.774269+0000" 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.778755+0000" 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.609 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.904726+0000" 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.610 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.782637+0000" 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.611 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.784402+0000" 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.612 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.613 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.786524+0000" 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.794605+0000" 2024-09-15T12:36:08.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.813714+0000" 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.616 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.900721+0000" 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.617 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 433, 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.818604+0000" 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.818619+0000" 2024-09-15T12:36:08.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.819048+0000" 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.819281+0000" 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.820876+0000" 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.621 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.907682+0000" 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.821751+0000" 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.822614+0000" 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.624 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.823753+0000" 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.625 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.626 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.835899+0000" 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.627 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.906956+0000" 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.628 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 431, 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.785840+0000" 2024-09-15T12:36:08.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.785866+0000" 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.786219+0000" 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.786394+0000" 2024-09-15T12:36:08.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.787800+0000" 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.632 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.914084+0000" 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.789208+0000" 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.790634+0000" 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.635 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.798618+0000" 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.637 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.807137+0000" 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.825908+0000" 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.912844+0000" 2024-09-15T12:36:08.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 443, 2024-09-15T12:36:08.640 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883449+0000" 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883461+0000" 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.641 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883751+0000" 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.642 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.883910+0000" 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.885270+0000" 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.643 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.937421+0000" 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.885807+0000" 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886055+0000" 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.886522+0000" 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.891784+0000" 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.650 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.935284+0000" 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 452, 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.930529+0000" 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.930541+0000" 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.930757+0000" 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.931007+0000" 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.932206+0000" 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.654 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.002417+0000" 2024-09-15T12:36:08.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.933062+0000" 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.933294+0000" 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.933777+0000" 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.660 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:07.938720+0000" 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.001174+0000" 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.662 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client_request", 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 468, 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "detail": { 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::StartEvent", 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.046258+0000" 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitActive", 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.046274+0000" 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.663 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::AwaitMap", 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.047938+0000" 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker", 2024-09-15T12:36:08.664 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ConnectionPipeline::GetPGMapping", 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.048414+0000" 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.665 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PerShardPipeline::CreateOrWaitPG", 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.051758+0000" 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGMap::PGCreationState", 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.666 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.116658+0000" 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::AwaitMap", 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.052247+0000" 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)1>::OSDMapBlocker", 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::OSDMapBlocker>::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitForActive", 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.052450+0000" 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::PGActivationBlocker", 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::RecoverMissing", 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.669 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::scrub::PGScrubber", 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::CheckAlreadyCompleteGetObc", 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.052892+0000" 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::GetOBC", 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::LockOBC", 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::Process", 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.055029+0000" 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::CommonPGPipeline::WaitRepop", 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.672 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "blocking_event": { 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::ClientRequest::PGPipeline::SendReply", 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "2024-09-15T12:36:08.115656+0000" 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::BlockerT::BlockingEvent::ExitBarrierEvent", 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "time_event": { 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crimson::osd::TrackableOperationT::CompletionEvent", 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: "initiated_at": "0.000000" 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.673 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-15T12:36:08.674 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-15T12:36:08.674 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:36:08.883 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:09.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:09.033+0000 7fa6d16bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:09.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:09.041+0000 7fa6d16bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:11.456 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:11.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:11.606+0000 7fa9fb591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:11.617 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:11.615+0000 7fa9fb591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:13.675 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 0, 1] 2024-09-15T12:36:13.676 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:36:13.676 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:13.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:13.805+0000 7f50da511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:13.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:13.817+0000 7f50da511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:14.061 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:14.104 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:14.105 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:14.128 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-09-15T12:36:12.198944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330105395,"num_objects":5031,"num_object_clones":0,"num_object_copies":15093,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":5107,"num_write_kb":322900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4128768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5083,"ondisk_log_size":5083,"up":67,"acting":67,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":261448,"kb_used_data":53384,"kb_used_omap":0,"kb_used_meta":208000,"kb_avail":377225912,"statfs":{"total":386547056640,"available":386279333888,"internally_reserved":0,"allocated":54665216,"data_stored":800455956,"data_compressed":37923317,"data_compressed_allocated":49922048,"data_compressed_original":798507008,"omap_allocated":0,"internal_metadata":212992000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001851"},"pg_stats":[{"pgid":"7.f","version":"142'361","reported_seq":405,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:32.178863+0000","last_change":"2024-09-15T12:35:27.897963+0000","last_active":"2024-09-15T12:35:32.178863+0000","last_peered":"2024-09-15T12:35:32.178863+0000","last_clean":"2024-09-15T12:34:58.629792+0000","last_became_active":"2024-09-15T12:35:27.897963+0000","last_became_peered":"2024-09-15T12:35:27.897963+0000","last_unstale":"2024-09-15T12:35:32.178863+0000","last_undegraded":"2024-09-15T12:35:27.806088+0000","last_fullsized":"2024-09-15T12:35:27.804376+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23658496,"num_objects":361,"num_object_clones":0,"num_object_copies":1083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":361,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":361}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"140'310","reported_seq":347,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.533850+0000","last_change":"2024-09-15T12:34:43.579635+0000","last_active":"2024-09-15T12:34:58.533850+0000","last_peered":"2024-09-15T12:34:58.533850+0000","last_clean":"2024-09-15T12:34:58.533850+0000","last_became_active":"2024-09-15T12:34:43.396238+0000","last_became_peered":"2024-09-15T12:34:43.396238+0000","last_unstale":"2024-09-15T12:36:01.764535+0000","last_undegraded":"2024-09-15T12:34:58.533850+0000","last_fullsized":"2024-09-15T12:34:58.533850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":19840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"142'343","reported_seq":382,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.789187+0000","last_change":"2024-09-15T12:34:10.863521+0000","last_active":"2024-09-15T12:35:47.789187+0000","last_peered":"2024-09-15T12:35:47.789187+0000","last_clean":"2024-09-15T12:35:47.789187+0000","last_became_active":"2024-09-15T12:34:10.858524+0000","last_became_peered":"2024-09-15T12:34:10.858524+0000","last_unstale":"2024-09-15T12:35:47.789187+0000","last_undegraded":"2024-09-15T12:35:47.789187+0000","last_fullsized":"2024-09-15T12:35:47.789187+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":343,"num_object_clones":0,"num_object_copies":1029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":343,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"142'336","reported_seq":375,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.788720+0000","last_change":"2024-09-15T12:34:10.918345+0000","last_active":"2024-09-15T12:35:47.788720+0000","last_peered":"2024-09-15T12:35:47.788720+0000","last_clean":"2024-09-15T12:35:47.788720+0000","last_became_active":"2024-09-15T12:34:10.908765+0000","last_became_peered":"2024-09-15T12:34:10.908765+0000","last_unstale":"2024-09-15T12:35:47.788720+0000","last_undegraded":"2024-09-15T12:35:47.788720+0000","last_fullsized":"2024-09-15T12:35:47.788720+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22020096,"num_objects":336,"num_object_clones":0,"num_object_copies":1008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":336,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"142'350","reported_seq":392,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:32.071948+0000","last_change":"2024-09-15T12:35:27.888850+0000","last_active":"2024-09-15T12:35:32.071948+0000","last_peered":"2024-09-15T12:35:32.071948+0000","last_clean":"2024-09-15T12:34:58.587173+0000","last_became_active":"2024-09-15T12:35:27.888850+0000","last_became_peered":"2024-09-15T12:35:27.888850+0000","last_unstale":"2024-09-15T12:35:32.071948+0000","last_undegraded":"2024-09-15T12:35:27.811033+0000","last_fullsized":"2024-09-15T12:35:27.809260+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22937600,"num_objects":350,"num_object_clones":0,"num_object_copies":1050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":350}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"142'289","reported_seq":329,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:31.918745+0000","last_change":"2024-09-15T12:35:27.894931+0000","last_active":"2024-09-15T12:35:31.918745+0000","last_peered":"2024-09-15T12:35:31.918745+0000","last_clean":"2024-09-15T12:34:58.383619+0000","last_became_active":"2024-09-15T12:35:27.894931+0000","last_became_peered":"2024-09-15T12:35:27.894931+0000","last_unstale":"2024-09-15T12:35:31.918745+0000","last_undegraded":"2024-09-15T12:35:27.810163+0000","last_fullsized":"2024-09-15T12:35:27.808361+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":289,"num_object_clones":0,"num_object_copies":867,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":289}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"142'327","reported_seq":366,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.791491+0000","last_change":"2024-09-15T12:34:10.947028+0000","last_active":"2024-09-15T12:35:47.791491+0000","last_peered":"2024-09-15T12:35:47.791491+0000","last_clean":"2024-09-15T12:35:47.791491+0000","last_became_active":"2024-09-15T12:34:10.934343+0000","last_became_peered":"2024-09-15T12:34:10.934343+0000","last_unstale":"2024-09-15T12:35:47.791491+0000","last_undegraded":"2024-09-15T12:35:47.791491+0000","last_fullsized":"2024-09-15T12:35:47.791491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"140'305","reported_seq":338,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.636058+0000","last_change":"2024-09-15T12:34:43.517925+0000","last_active":"2024-09-15T12:34:58.636058+0000","last_peered":"2024-09-15T12:34:58.636058+0000","last_clean":"2024-09-15T12:34:58.636058+0000","last_became_active":"2024-09-15T12:34:43.358315+0000","last_became_peered":"2024-09-15T12:34:43.358315+0000","last_unstale":"2024-09-15T12:36:01.764537+0000","last_undegraded":"2024-09-15T12:34:58.636058+0000","last_fullsized":"2024-09-15T12:34:58.636058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":305,"num_object_clones":0,"num_object_copies":915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"142'344","reported_seq":379,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:32.197877+0000","last_change":"2024-09-15T12:35:27.857376+0000","last_active":"2024-09-15T12:35:32.197877+0000","last_peered":"2024-09-15T12:35:32.197877+0000","last_clean":"2024-09-15T12:34:58.569188+0000","last_became_active":"2024-09-15T12:35:27.857376+0000","last_became_peered":"2024-09-15T12:35:27.857376+0000","last_unstale":"2024-09-15T12:35:32.197877+0000","last_undegraded":"2024-09-15T12:35:27.800869+0000","last_fullsized":"2024-09-15T12:35:27.799465+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22544384,"num_objects":344,"num_object_clones":0,"num_object_copies":1032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":344}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":316,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:27.872692+0000","last_change":"2024-09-15T12:35:27.872045+0000","last_active":"2024-09-15T12:35:27.872692+0000","last_peered":"2024-09-15T12:35:27.872692+0000","last_clean":"2024-09-15T12:34:43.801852+0000","last_became_active":"2024-09-15T12:35:27.872045+0000","last_became_peered":"2024-09-15T12:35:27.872045+0000","last_unstale":"2024-09-15T12:35:27.872692+0000","last_undegraded":"2024-09-15T12:35:27.796016+0000","last_fullsized":"2024-09-15T12:35:27.794565+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"140'307","reported_seq":334,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.595593+0000","last_change":"2024-09-15T12:34:10.905570+0000","last_active":"2024-09-15T12:34:58.595593+0000","last_peered":"2024-09-15T12:34:58.595593+0000","last_clean":"2024-09-15T12:34:58.595593+0000","last_became_active":"2024-09-15T12:34:10.888754+0000","last_became_peered":"2024-09-15T12:34:10.888754+0000","last_unstale":"2024-09-15T12:36:01.764543+0000","last_undegraded":"2024-09-15T12:34:58.595593+0000","last_fullsized":"2024-09-15T12:34:58.595593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":307,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":379,"reported_epoch":134,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:10.782396+0000","last_change":"2024-09-15T12:34:04.793491+0000","last_active":"2024-09-15T12:34:10.782396+0000","last_peered":"2024-09-15T12:34:10.782396+0000","last_clean":"2024-09-15T12:34:10.782396+0000","last_became_active":"2024-09-15T12:34:03.898901+0000","last_became_peered":"2024-09-15T12:34:03.898901+0000","last_unstale":"2024-09-15T12:34:41.231947+0000","last_undegraded":"2024-09-15T12:34:10.782396+0000","last_fullsized":"2024-09-15T12:34:10.782396+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"142'326","reported_seq":480,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:48.901203+0000","last_change":"2024-09-15T12:35:48.902283+0000","last_active":"2024-09-15T12:35:48.901203+0000","last_peered":"2024-09-15T12:35:48.901203+0000","last_clean":"2024-09-15T12:34:58.633762+0000","last_became_active":"2024-09-15T12:35:47.879055+0000","last_became_peered":"2024-09-15T12:35:47.879055+0000","last_unstale":"2024-09-15T12:35:48.901203+0000","last_undegraded":"2024-09-15T12:35:48.901203+0000","last_fullsized":"2024-09-15T12:35:48.901203+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21364736,"num_objects":326,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":326,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"140'314","reported_seq":342,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.667827+0000","last_change":"2024-09-15T12:34:43.399802+0000","last_active":"2024-09-15T12:34:58.667827+0000","last_peered":"2024-09-15T12:34:58.667827+0000","last_clean":"2024-09-15T12:34:58.667827+0000","last_became_active":"2024-09-15T12:34:43.394245+0000","last_became_peered":"2024-09-15T12:34:43.394245+0000","last_unstale":"2024-09-15T12:36:01.764547+0000","last_undegraded":"2024-09-15T12:34:58.667827+0000","last_fullsized":"2024-09-15T12:34:58.667827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":381,"reported_epoch":134,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:10.774887+0000","last_change":"2024-09-15T12:34:04.773471+0000","last_active":"2024-09-15T12:34:10.774887+0000","last_peered":"2024-09-15T12:34:10.774887+0000","last_clean":"2024-09-15T12:34:10.774887+0000","last_became_active":"2024-09-15T12:34:03.857002+0000","last_became_peered":"2024-09-15T12:34:03.857002+0000","last_unstale":"2024-09-15T12:34:41.231950+0000","last_undegraded":"2024-09-15T12:34:10.774887+0000","last_fullsized":"2024-09-15T12:34:10.774887+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:10.876621+0000","last_change":"2024-09-15T12:34:10.877777+0000","last_active":"2024-09-15T12:34:10.876621+0000","last_peered":"2024-09-15T12:34:10.876621+0000","last_clean":"2024-09-15T12:34:09.748678+0000","last_became_active":"2024-09-15T12:34:10.872681+0000","last_became_peered":"2024-09-15T12:34:10.872681+0000","last_unstale":"2024-09-15T12:34:41.231967+0000","last_undegraded":"2024-09-15T12:34:10.876621+0000","last_fullsized":"2024-09-15T12:34:10.876621+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":399,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.859045+0000","last_change":"2024-09-15T12:35:47.860218+0000","last_active":"2024-09-15T12:35:47.859045+0000","last_peered":"2024-09-15T12:35:47.859045+0000","last_clean":"2024-09-15T12:34:43.787911+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:35:47.859045+0000","last_undegraded":"2024-09-15T12:35:47.859045+0000","last_fullsized":"2024-09-15T12:35:47.859045+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"142'374","reported_seq":413,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.791034+0000","last_change":"2024-09-15T12:34:10.950546+0000","last_active":"2024-09-15T12:35:47.791034+0000","last_peered":"2024-09-15T12:35:47.791034+0000","last_clean":"2024-09-15T12:35:47.791034+0000","last_became_active":"2024-09-15T12:34:10.940150+0000","last_became_peered":"2024-09-15T12:34:10.940150+0000","last_unstale":"2024-09-15T12:35:47.791034+0000","last_undegraded":"2024-09-15T12:35:47.791034+0000","last_fullsized":"2024-09-15T12:35:47.791034+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":374,"num_object_clones":0,"num_object_copies":1122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":374,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":254,"reported_epoch":142,"state":"active+undersized","last_fresh":"2024-09-15T12:35:27.888584+0000","last_change":"2024-09-15T12:35:27.887465+0000","last_active":"2024-09-15T12:35:27.888584+0000","last_peered":"2024-09-15T12:35:27.888584+0000","last_clean":"2024-09-15T12:34:43.807525+0000","last_became_active":"2024-09-15T12:35:27.887465+0000","last_became_peered":"2024-09-15T12:35:27.887465+0000","last_unstale":"2024-09-15T12:35:27.888584+0000","last_undegraded":"2024-09-15T12:35:27.888584+0000","last_fullsized":"2024-09-15T12:35:27.804168+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"142'373","reported_seq":408,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:32.122993+0000","last_change":"2024-09-15T12:35:27.856224+0000","last_active":"2024-09-15T12:35:32.122993+0000","last_peered":"2024-09-15T12:35:32.122993+0000","last_clean":"2024-09-15T12:34:58.699364+0000","last_became_active":"2024-09-15T12:35:27.856224+0000","last_became_peered":"2024-09-15T12:35:27.856224+0000","last_unstale":"2024-09-15T12:35:32.122993+0000","last_undegraded":"2024-09-15T12:35:27.791405+0000","last_fullsized":"2024-09-15T12:35:27.790196+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":373,"num_object_clones":0,"num_object_copies":1119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":373,"num_write_kb":23872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":373}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":399,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-15T12:35:47.863901+0000","last_change":"2024-09-15T12:35:47.865153+0000","last_active":"2024-09-15T12:35:47.863901+0000","last_peered":"2024-09-15T12:35:47.863901+0000","last_clean":"2024-09-15T12:34:43.787512+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:35:47.863901+0000","last_undegraded":"2024-09-15T12:35:47.863901+0000","last_fullsized":"2024-09-15T12:35:47.863901+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"142'369","reported_seq":409,"reported_epoch":142,"state":"active+undersized+degraded","last_fresh":"2024-09-15T12:35:32.160744+0000","last_change":"2024-09-15T12:35:27.873790+0000","last_active":"2024-09-15T12:35:32.160744+0000","last_peered":"2024-09-15T12:35:32.160744+0000","last_clean":"2024-09-15T12:34:58.726476+0000","last_became_active":"2024-09-15T12:35:27.873790+0000","last_became_peered":"2024-09-15T12:35:27.873790+0000","last_unstale":"2024-09-15T12:35:32.160744+0000","last_undegraded":"2024-09-15T12:35:27.802388+0000","last_fullsized":"2024-09-15T12:35:27.800641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24182784,"num_objects":369,"num_object_clones":0,"num_object_copies":1107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":369,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":369,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":369}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":329515008,"num_objects":5028,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5028,"num_write_kb":321792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":4128768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49799168,"data_stored":796786688,"data_compressed":37833149,"data_compressed_allocated":49799168,"data_compressed_original":796786688,"omap_allocated":0,"internal_metadata":0},"log_size":5028,"ondisk_log_size":5028,"up":42,"acting":42,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":23,"acting":23,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82976,"kb_used_data":22032,"kb_used_omap":0,"kb_used_meta":60928,"kb_avail":94288864,"statfs":{"total":96636764160,"available":96551796736,"internally_reserved":0,"allocated":22560768,"data_stored":342121179,"data_compressed":16223863,"data_compressed_allocated":21356544,"data_compressed_original":341622784,"omap_allocated":0,"internal_metadata":62390272},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860685,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":57236,"kb_used_data":13636,"kb_used_omap":0,"kb_used_meta":43584,"kb_avail":94314604,"statfs":{"total":96636764160,"available":96578154496,"internally_reserved":0,"allocated":13963264,"data_stored":204240645,"data_compressed":9676676,"data_compressed_allocated":12738560,"data_compressed_original":203735040,"omap_allocated":0,"internal_metadata":44630016},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":38,"seq":163208757817,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64404,"kb_used_data":1156,"kb_used_omap":0,"kb_used_meta":63232,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1183744,"data_stored":1043863,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":64749568},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15785984,"data_stored":252575744,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12697600,"data_stored":203161600,"data_compressed":9646620,"data_compressed_allocated":12697600,"data_compressed_original":203161600,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":21315584,"data_stored":341049344,"data_compressed":16193807,"data_compressed_allocated":21315584,"data_compressed_original":341049344,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:14.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 7.a 7.6 7.2 2024-09-15T12:36:14.226 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:14.220+0000 7f3b2a311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:14.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:14.233+0000 7f3b2a311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:14.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:14.329+0000 7f01ad591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:14.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:14.345+0000 7f01ad591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:14.598 INFO:teuthology.orchestra.run.smithi103.stderr:instructing pg(s) [7.2,7.6,7.a] on osd.3 to force-recovery; 2024-09-15T12:36:14.598 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:36:16.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:16.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:16.747+0000 7fd95c791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:16.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:16.761+0000 7fd95c791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.037 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:19.201 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.196+0000 7fc887046640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.205+0000 7fc887046640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.616 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 0, 1] 2024-09-15T12:36:19.617 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:36:19.617 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:36:19.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:36:19.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.767+0000 7f7b6fc57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.781+0000 7f7b6fc57640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:19.817 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:19.818 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:19.820 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:19.821 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:19.821 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:36:19.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.980+0000 7ffbc486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:19.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:19.998+0000 7ffbc486c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:20.042 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:20.043 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:20.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:20.046 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:20.047 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:36:20.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:20.198+0000 7eff35f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:20.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:20.217+0000 7eff35f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:20.248 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:20.249 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:20.253 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:20.253 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:20.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:36:20.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:20.409+0000 7f974b8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:20.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:20.423+0000 7f974b8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:20.457 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:20.458 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:20.462 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:20.462 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:21.544 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:21.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:21.693+0000 7f0d508ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:21.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:21.701+0000 7f0d508ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:23.962 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:24.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:24.117+0000 7f2593b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:24.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:24.127+0000 7f2593b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:25.463 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [2, 3, 0, 1] 2024-09-15T12:36:25.464 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T12:36:25.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:25.630+0000 7fd84a151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:25.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:25.651+0000 7fd84a151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:26.115 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T12:36:26.135 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T12:36:26.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:26.288+0000 7fafb6591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:26.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:26.305+0000 7fafb6591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:26.487 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:26.633+0000 7f78216bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:26.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:26.643+0000 7f78216bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:27.130 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T12:36:27.156 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T12:36:27.330 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:27.315+0000 7fcf21d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:27.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:27.333+0000 7fcf21d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T12:36:28.151 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T12:36:28.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:28.300+0000 7eff74f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:28.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:28.318+0000 7eff74f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:28.769 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T12:36:28.792 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T12:36:28.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:28.935 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T12:36:28.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:28.937+0000 7fc0deb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:28.955 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:28.953+0000 7fc0deb51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.091 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.086+0000 7efc13791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.098 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.096+0000 7efc13791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.313 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:29.314 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:29.338 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":578,"stamp":"2024-09-15T12:36:28.768501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185467456,"num_objects":2825,"num_object_clones":0,"num_object_copies":8475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":7954,"num_write_kb":343380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7935,"ondisk_log_size":7935,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214776,"kb_used_data":42616,"kb_used_omap":0,"kb_used_meta":172096,"kb_avail":377272584,"statfs":{"total":386547056640,"available":386327126016,"internally_reserved":0,"allocated":43638784,"data_stored":620713220,"data_compressed":29385610,"data_compressed_allocated":38682624,"data_compressed_original":618676224,"omap_allocated":0,"internal_metadata":176226304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-97779688,"num_objects":-1491,"num_object_clones":0,"num_object_copies":-4473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1493,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":131096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.566663"},"pg_stats":[{"pgid":"7.f","version":"148'523","reported_seq":720,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.550626+0000","last_change":"2024-09-15T12:36:13.515356+0000","last_active":"2024-09-15T12:36:15.550626+0000","last_peered":"2024-09-15T12:36:15.550626+0000","last_clean":"2024-09-15T12:36:15.550626+0000","last_became_active":"2024-09-15T12:36:05.995918+0000","last_became_peered":"2024-09-15T12:36:05.995918+0000","last_unstale":"2024-09-15T12:36:15.550626+0000","last_undegraded":"2024-09-15T12:36:15.550626+0000","last_fullsized":"2024-09-15T12:36:15.550626+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"140'310","reported_seq":347,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.533850+0000","last_change":"2024-09-15T12:34:43.579635+0000","last_active":"2024-09-15T12:34:58.533850+0000","last_peered":"2024-09-15T12:34:58.533850+0000","last_clean":"2024-09-15T12:34:58.533850+0000","last_became_active":"2024-09-15T12:34:43.396238+0000","last_became_peered":"2024-09-15T12:34:43.396238+0000","last_unstale":"2024-09-15T12:36:01.764535+0000","last_undegraded":"2024-09-15T12:34:58.533850+0000","last_fullsized":"2024-09-15T12:34:58.533850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":19840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"149'636","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.279506+0000","last_change":"2024-09-15T12:36:10.041250+0000","last_active":"2024-09-15T12:36:26.279506+0000","last_peered":"2024-09-15T12:36:26.279506+0000","last_clean":"2024-09-15T12:36:26.279506+0000","last_became_active":"2024-09-15T12:36:05.959397+0000","last_became_peered":"2024-09-15T12:36:05.959397+0000","last_unstale":"2024-09-15T12:36:26.279506+0000","last_undegraded":"2024-09-15T12:36:26.279506+0000","last_fullsized":"2024-09-15T12:36:26.279506+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342360,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":636,"num_write_kb":21953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"149'629","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.386200+0000","last_change":"2024-09-15T12:36:10.492150+0000","last_active":"2024-09-15T12:36:26.386200+0000","last_peered":"2024-09-15T12:36:26.386200+0000","last_clean":"2024-09-15T12:36:26.386200+0000","last_became_active":"2024-09-15T12:36:05.941034+0000","last_became_peered":"2024-09-15T12:36:05.941034+0000","last_unstale":"2024-09-15T12:36:26.386200+0000","last_undegraded":"2024-09-15T12:36:26.386200+0000","last_fullsized":"2024-09-15T12:36:26.386200+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":629,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"148'491","reported_seq":711,"reported_epoch":148,"state":"active+recovering+degraded","last_fresh":"2024-09-15T12:36:15.611672+0000","last_change":"2024-09-15T12:36:09.103977+0000","last_active":"2024-09-15T12:36:15.611672+0000","last_peered":"2024-09-15T12:36:15.611672+0000","last_clean":"2024-09-15T12:34:58.587173+0000","last_became_active":"2024-09-15T12:36:05.989572+0000","last_became_peered":"2024-09-15T12:36:05.989572+0000","last_unstale":"2024-09-15T12:36:15.611672+0000","last_undegraded":"2024-09-15T12:36:05.895696+0000","last_fullsized":"2024-09-15T12:36:15.611672+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":491,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"0,1,3","objects":204},{"shards":"1,3","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"148'411","reported_seq":565,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.438511+0000","last_change":"2024-09-15T12:35:48.808494+0000","last_active":"2024-09-15T12:36:15.438511+0000","last_peered":"2024-09-15T12:36:15.438511+0000","last_clean":"2024-09-15T12:36:15.438511+0000","last_became_active":"2024-09-15T12:35:47.887493+0000","last_became_peered":"2024-09-15T12:35:47.887493+0000","last_unstale":"2024-09-15T12:36:15.438511+0000","last_undegraded":"2024-09-15T12:36:15.438511+0000","last_fullsized":"2024-09-15T12:36:15.438511+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10944512,"num_objects":167,"num_object_clones":0,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":411,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3014656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"149'611","reported_seq":696,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.272815+0000","last_change":"2024-09-15T12:36:09.540255+0000","last_active":"2024-09-15T12:36:26.272815+0000","last_peered":"2024-09-15T12:36:26.272815+0000","last_clean":"2024-09-15T12:36:26.272815+0000","last_became_active":"2024-09-15T12:36:05.980983+0000","last_became_peered":"2024-09-15T12:36:05.980983+0000","last_unstale":"2024-09-15T12:36:26.272815+0000","last_undegraded":"2024-09-15T12:36:26.272815+0000","last_fullsized":"2024-09-15T12:36:26.272815+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"140'305","reported_seq":338,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.636058+0000","last_change":"2024-09-15T12:34:43.517925+0000","last_active":"2024-09-15T12:34:58.636058+0000","last_peered":"2024-09-15T12:34:58.636058+0000","last_clean":"2024-09-15T12:34:58.636058+0000","last_became_active":"2024-09-15T12:34:43.358315+0000","last_became_peered":"2024-09-15T12:34:43.358315+0000","last_unstale":"2024-09-15T12:36:01.764537+0000","last_undegraded":"2024-09-15T12:34:58.636058+0000","last_fullsized":"2024-09-15T12:34:58.636058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":305,"num_object_clones":0,"num_object_copies":915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"148'492","reported_seq":693,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.570505+0000","last_change":"2024-09-15T12:36:07.374317+0000","last_active":"2024-09-15T12:36:15.570505+0000","last_peered":"2024-09-15T12:36:15.570505+0000","last_clean":"2024-09-15T12:36:15.570505+0000","last_became_active":"2024-09-15T12:36:05.944934+0000","last_became_peered":"2024-09-15T12:36:05.944934+0000","last_unstale":"2024-09-15T12:36:15.570505+0000","last_undegraded":"2024-09-15T12:36:15.570505+0000","last_fullsized":"2024-09-15T12:36:15.570505+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":3276800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"140'307","reported_seq":334,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.595593+0000","last_change":"2024-09-15T12:34:10.905570+0000","last_active":"2024-09-15T12:34:58.595593+0000","last_peered":"2024-09-15T12:34:58.595593+0000","last_clean":"2024-09-15T12:34:58.595593+0000","last_became_active":"2024-09-15T12:34:10.888754+0000","last_became_peered":"2024-09-15T12:34:10.888754+0000","last_unstale":"2024-09-15T12:36:01.764543+0000","last_undegraded":"2024-09-15T12:34:58.595593+0000","last_fullsized":"2024-09-15T12:34:58.595593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":307,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"149'611","reported_seq":780,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.250859+0000","last_change":"2024-09-15T12:35:48.902283+0000","last_active":"2024-09-15T12:36:26.250859+0000","last_peered":"2024-09-15T12:36:26.250859+0000","last_clean":"2024-09-15T12:36:26.250859+0000","last_became_active":"2024-09-15T12:35:47.879055+0000","last_became_peered":"2024-09-15T12:35:47.879055+0000","last_unstale":"2024-09-15T12:36:26.250859+0000","last_undegraded":"2024-09-15T12:36:26.250859+0000","last_fullsized":"2024-09-15T12:36:26.250859+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"140'314","reported_seq":342,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.667827+0000","last_change":"2024-09-15T12:34:43.399802+0000","last_active":"2024-09-15T12:34:58.667827+0000","last_peered":"2024-09-15T12:34:58.667827+0000","last_clean":"2024-09-15T12:34:58.667827+0000","last_became_active":"2024-09-15T12:34:43.394245+0000","last_became_peered":"2024-09-15T12:34:43.394245+0000","last_unstale":"2024-09-15T12:36:01.764547+0000","last_undegraded":"2024-09-15T12:34:58.667827+0000","last_fullsized":"2024-09-15T12:34:58.667827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"148'471","reported_seq":605,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:16.995834+0000","last_change":"2024-09-15T12:36:08.724927+0000","last_active":"2024-09-15T12:36:16.995834+0000","last_peered":"2024-09-15T12:36:16.995834+0000","last_clean":"2024-09-15T12:36:16.995834+0000","last_became_active":"2024-09-15T12:36:05.941883+0000","last_became_peered":"2024-09-15T12:36:05.941883+0000","last_unstale":"2024-09-15T12:36:16.995834+0000","last_undegraded":"2024-09-15T12:36:16.995834+0000","last_fullsized":"2024-09-15T12:36:16.995834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11337728,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"149'712","reported_seq":803,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.230995+0000","last_change":"2024-09-15T12:36:09.002505+0000","last_active":"2024-09-15T12:36:26.230995+0000","last_peered":"2024-09-15T12:36:26.230995+0000","last_clean":"2024-09-15T12:36:26.230995+0000","last_became_active":"2024-09-15T12:36:05.965072+0000","last_became_peered":"2024-09-15T12:36:05.965072+0000","last_unstale":"2024-09-15T12:36:26.230995+0000","last_undegraded":"2024-09-15T12:36:26.230995+0000","last_fullsized":"2024-09-15T12:36:26.230995+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"148'537","reported_seq":712,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.510327+0000","last_change":"2024-09-15T12:35:49.003540+0000","last_active":"2024-09-15T12:36:15.510327+0000","last_peered":"2024-09-15T12:36:15.510327+0000","last_clean":"2024-09-15T12:36:15.510327+0000","last_became_active":"2024-09-15T12:35:47.873977+0000","last_became_peered":"2024-09-15T12:35:47.873977+0000","last_unstale":"2024-09-15T12:36:15.510327+0000","last_undegraded":"2024-09-15T12:36:15.510327+0000","last_fullsized":"2024-09-15T12:36:15.510327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":537,"num_write_kb":23872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"148'520","reported_seq":718,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.420717+0000","last_change":"2024-09-15T12:36:14.870404+0000","last_active":"2024-09-15T12:36:15.420717+0000","last_peered":"2024-09-15T12:36:15.420717+0000","last_clean":"2024-09-15T12:36:15.420717+0000","last_became_active":"2024-09-15T12:36:05.958461+0000","last_became_peered":"2024-09-15T12:36:05.958461+0000","last_unstale":"2024-09-15T12:36:15.420717+0000","last_undegraded":"2024-09-15T12:36:15.420717+0000","last_fullsized":"2024-09-15T12:36:15.420717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":184877080,"num_objects":2822,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7876,"num_write_kb":342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37810176,"data_stored":604831792,"data_compressed":28719722,"data_compressed_allocated":37801984,"data_compressed_original":604831744,"omap_allocated":0,"internal_metadata":0},"log_size":7880,"ondisk_log_size":7880,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8028160,"data_stored":128450560,"data_compressed":6099520,"data_compressed_allocated":8028160,"data_compressed_original":128450560,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15785984,"data_stored":252575744,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":24510488,"data_compressed":1163888,"data_compressed_allocated":1531904,"data_compressed_original":24510464,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12460032,"data_stored":199295000,"data_compressed":9463592,"data_compressed_allocated":12455936,"data_compressed_original":199294976,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:29.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:29.424 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Removed 5577 objects 2024-09-15T12:36:29.424 INFO:tasks.radosbench.radosbench.0.smithi103.stdout:Clean up completed and total clean up time :25.5163 2024-09-15T12:36:29.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.497+0000 7f77fd308640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.505+0000 7f77fd308640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.694 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:29.695 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:29.707 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":578,"stamp":"2024-09-15T12:36:28.768501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185467456,"num_objects":2825,"num_object_clones":0,"num_object_copies":8475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":7954,"num_write_kb":343380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7935,"ondisk_log_size":7935,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214776,"kb_used_data":42616,"kb_used_omap":0,"kb_used_meta":172096,"kb_avail":377272584,"statfs":{"total":386547056640,"available":386327126016,"internally_reserved":0,"allocated":43638784,"data_stored":620713220,"data_compressed":29385610,"data_compressed_allocated":38682624,"data_compressed_original":618676224,"omap_allocated":0,"internal_metadata":176226304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-97779688,"num_objects":-1491,"num_object_clones":0,"num_object_copies":-4473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1493,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":131096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.566663"},"pg_stats":[{"pgid":"7.f","version":"148'523","reported_seq":720,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.550626+0000","last_change":"2024-09-15T12:36:13.515356+0000","last_active":"2024-09-15T12:36:15.550626+0000","last_peered":"2024-09-15T12:36:15.550626+0000","last_clean":"2024-09-15T12:36:15.550626+0000","last_became_active":"2024-09-15T12:36:05.995918+0000","last_became_peered":"2024-09-15T12:36:05.995918+0000","last_unstale":"2024-09-15T12:36:15.550626+0000","last_undegraded":"2024-09-15T12:36:15.550626+0000","last_fullsized":"2024-09-15T12:36:15.550626+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"140'310","reported_seq":347,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.533850+0000","last_change":"2024-09-15T12:34:43.579635+0000","last_active":"2024-09-15T12:34:58.533850+0000","last_peered":"2024-09-15T12:34:58.533850+0000","last_clean":"2024-09-15T12:34:58.533850+0000","last_became_active":"2024-09-15T12:34:43.396238+0000","last_became_peered":"2024-09-15T12:34:43.396238+0000","last_unstale":"2024-09-15T12:36:01.764535+0000","last_undegraded":"2024-09-15T12:34:58.533850+0000","last_fullsized":"2024-09-15T12:34:58.533850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":19840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"149'636","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.279506+0000","last_change":"2024-09-15T12:36:10.041250+0000","last_active":"2024-09-15T12:36:26.279506+0000","last_peered":"2024-09-15T12:36:26.279506+0000","last_clean":"2024-09-15T12:36:26.279506+0000","last_became_active":"2024-09-15T12:36:05.959397+0000","last_became_peered":"2024-09-15T12:36:05.959397+0000","last_unstale":"2024-09-15T12:36:26.279506+0000","last_undegraded":"2024-09-15T12:36:26.279506+0000","last_fullsized":"2024-09-15T12:36:26.279506+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342360,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":636,"num_write_kb":21953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"149'629","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.386200+0000","last_change":"2024-09-15T12:36:10.492150+0000","last_active":"2024-09-15T12:36:26.386200+0000","last_peered":"2024-09-15T12:36:26.386200+0000","last_clean":"2024-09-15T12:36:26.386200+0000","last_became_active":"2024-09-15T12:36:05.941034+0000","last_became_peered":"2024-09-15T12:36:05.941034+0000","last_unstale":"2024-09-15T12:36:26.386200+0000","last_undegraded":"2024-09-15T12:36:26.386200+0000","last_fullsized":"2024-09-15T12:36:26.386200+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":629,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"148'491","reported_seq":711,"reported_epoch":148,"state":"active+recovering+degraded","last_fresh":"2024-09-15T12:36:15.611672+0000","last_change":"2024-09-15T12:36:09.103977+0000","last_active":"2024-09-15T12:36:15.611672+0000","last_peered":"2024-09-15T12:36:15.611672+0000","last_clean":"2024-09-15T12:34:58.587173+0000","last_became_active":"2024-09-15T12:36:05.989572+0000","last_became_peered":"2024-09-15T12:36:05.989572+0000","last_unstale":"2024-09-15T12:36:15.611672+0000","last_undegraded":"2024-09-15T12:36:05.895696+0000","last_fullsized":"2024-09-15T12:36:15.611672+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":491,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"0,1,3","objects":204},{"shards":"1,3","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"148'411","reported_seq":565,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.438511+0000","last_change":"2024-09-15T12:35:48.808494+0000","last_active":"2024-09-15T12:36:15.438511+0000","last_peered":"2024-09-15T12:36:15.438511+0000","last_clean":"2024-09-15T12:36:15.438511+0000","last_became_active":"2024-09-15T12:35:47.887493+0000","last_became_peered":"2024-09-15T12:35:47.887493+0000","last_unstale":"2024-09-15T12:36:15.438511+0000","last_undegraded":"2024-09-15T12:36:15.438511+0000","last_fullsized":"2024-09-15T12:36:15.438511+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10944512,"num_objects":167,"num_object_clones":0,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":411,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3014656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"149'611","reported_seq":696,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.272815+0000","last_change":"2024-09-15T12:36:09.540255+0000","last_active":"2024-09-15T12:36:26.272815+0000","last_peered":"2024-09-15T12:36:26.272815+0000","last_clean":"2024-09-15T12:36:26.272815+0000","last_became_active":"2024-09-15T12:36:05.980983+0000","last_became_peered":"2024-09-15T12:36:05.980983+0000","last_unstale":"2024-09-15T12:36:26.272815+0000","last_undegraded":"2024-09-15T12:36:26.272815+0000","last_fullsized":"2024-09-15T12:36:26.272815+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"140'305","reported_seq":338,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.636058+0000","last_change":"2024-09-15T12:34:43.517925+0000","last_active":"2024-09-15T12:34:58.636058+0000","last_peered":"2024-09-15T12:34:58.636058+0000","last_clean":"2024-09-15T12:34:58.636058+0000","last_became_active":"2024-09-15T12:34:43.358315+0000","last_became_peered":"2024-09-15T12:34:43.358315+0000","last_unstale":"2024-09-15T12:36:01.764537+0000","last_undegraded":"2024-09-15T12:34:58.636058+0000","last_fullsized":"2024-09-15T12:34:58.636058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":305,"num_object_clones":0,"num_object_copies":915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"148'492","reported_seq":693,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.570505+0000","last_change":"2024-09-15T12:36:07.374317+0000","last_active":"2024-09-15T12:36:15.570505+0000","last_peered":"2024-09-15T12:36:15.570505+0000","last_clean":"2024-09-15T12:36:15.570505+0000","last_became_active":"2024-09-15T12:36:05.944934+0000","last_became_peered":"2024-09-15T12:36:05.944934+0000","last_unstale":"2024-09-15T12:36:15.570505+0000","last_undegraded":"2024-09-15T12:36:15.570505+0000","last_fullsized":"2024-09-15T12:36:15.570505+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":3276800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"140'307","reported_seq":334,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.595593+0000","last_change":"2024-09-15T12:34:10.905570+0000","last_active":"2024-09-15T12:34:58.595593+0000","last_peered":"2024-09-15T12:34:58.595593+0000","last_clean":"2024-09-15T12:34:58.595593+0000","last_became_active":"2024-09-15T12:34:10.888754+0000","last_became_peered":"2024-09-15T12:34:10.888754+0000","last_unstale":"2024-09-15T12:36:01.764543+0000","last_undegraded":"2024-09-15T12:34:58.595593+0000","last_fullsized":"2024-09-15T12:34:58.595593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":307,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"149'611","reported_seq":780,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.250859+0000","last_change":"2024-09-15T12:35:48.902283+0000","last_active":"2024-09-15T12:36:26.250859+0000","last_peered":"2024-09-15T12:36:26.250859+0000","last_clean":"2024-09-15T12:36:26.250859+0000","last_became_active":"2024-09-15T12:35:47.879055+0000","last_became_peered":"2024-09-15T12:35:47.879055+0000","last_unstale":"2024-09-15T12:36:26.250859+0000","last_undegraded":"2024-09-15T12:36:26.250859+0000","last_fullsized":"2024-09-15T12:36:26.250859+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"140'314","reported_seq":342,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.667827+0000","last_change":"2024-09-15T12:34:43.399802+0000","last_active":"2024-09-15T12:34:58.667827+0000","last_peered":"2024-09-15T12:34:58.667827+0000","last_clean":"2024-09-15T12:34:58.667827+0000","last_became_active":"2024-09-15T12:34:43.394245+0000","last_became_peered":"2024-09-15T12:34:43.394245+0000","last_unstale":"2024-09-15T12:36:01.764547+0000","last_undegraded":"2024-09-15T12:34:58.667827+0000","last_fullsized":"2024-09-15T12:34:58.667827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"148'471","reported_seq":605,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:16.995834+0000","last_change":"2024-09-15T12:36:08.724927+0000","last_active":"2024-09-15T12:36:16.995834+0000","last_peered":"2024-09-15T12:36:16.995834+0000","last_clean":"2024-09-15T12:36:16.995834+0000","last_became_active":"2024-09-15T12:36:05.941883+0000","last_became_peered":"2024-09-15T12:36:05.941883+0000","last_unstale":"2024-09-15T12:36:16.995834+0000","last_undegraded":"2024-09-15T12:36:16.995834+0000","last_fullsized":"2024-09-15T12:36:16.995834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11337728,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"149'712","reported_seq":803,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.230995+0000","last_change":"2024-09-15T12:36:09.002505+0000","last_active":"2024-09-15T12:36:26.230995+0000","last_peered":"2024-09-15T12:36:26.230995+0000","last_clean":"2024-09-15T12:36:26.230995+0000","last_became_active":"2024-09-15T12:36:05.965072+0000","last_became_peered":"2024-09-15T12:36:05.965072+0000","last_unstale":"2024-09-15T12:36:26.230995+0000","last_undegraded":"2024-09-15T12:36:26.230995+0000","last_fullsized":"2024-09-15T12:36:26.230995+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"148'537","reported_seq":712,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.510327+0000","last_change":"2024-09-15T12:35:49.003540+0000","last_active":"2024-09-15T12:36:15.510327+0000","last_peered":"2024-09-15T12:36:15.510327+0000","last_clean":"2024-09-15T12:36:15.510327+0000","last_became_active":"2024-09-15T12:35:47.873977+0000","last_became_peered":"2024-09-15T12:35:47.873977+0000","last_unstale":"2024-09-15T12:36:15.510327+0000","last_undegraded":"2024-09-15T12:36:15.510327+0000","last_fullsized":"2024-09-15T12:36:15.510327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":537,"num_write_kb":23872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"148'520","reported_seq":718,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.420717+0000","last_change":"2024-09-15T12:36:14.870404+0000","last_active":"2024-09-15T12:36:15.420717+0000","last_peered":"2024-09-15T12:36:15.420717+0000","last_clean":"2024-09-15T12:36:15.420717+0000","last_became_active":"2024-09-15T12:36:05.958461+0000","last_became_peered":"2024-09-15T12:36:05.958461+0000","last_unstale":"2024-09-15T12:36:15.420717+0000","last_undegraded":"2024-09-15T12:36:15.420717+0000","last_fullsized":"2024-09-15T12:36:15.420717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":184877080,"num_objects":2822,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7876,"num_write_kb":342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37810176,"data_stored":604831792,"data_compressed":28719722,"data_compressed_allocated":37801984,"data_compressed_original":604831744,"omap_allocated":0,"internal_metadata":0},"log_size":7880,"ondisk_log_size":7880,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8028160,"data_stored":128450560,"data_compressed":6099520,"data_compressed_allocated":8028160,"data_compressed_original":128450560,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15785984,"data_stored":252575744,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":24510488,"data_compressed":1163888,"data_compressed_allocated":1531904,"data_compressed_original":24510464,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12460032,"data_stored":199295000,"data_compressed":9463592,"data_compressed_allocated":12455936,"data_compressed_original":199294976,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:29.708 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:36:29.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.850+0000 7f81026d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:29.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:29.857+0000 7f81026d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:30.151 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:30.151 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5/8475 objects degraded (0.059%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":876,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":152,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403764,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"stale+active+clean","count":7},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2825,"data_bytes":185467456,"bytes_used":219930624,"bytes_avail":386327126016,"bytes_total":386547056640,"degraded_objects":5,"degraded_total":8475,"degraded_ratio":0.00058997050147492625,"recovering_objects_per_sec":6,"recovering_bytes_per_sec":21844,"recovering_keys_per_sec":0,"num_objects_recovered":40,"num_bytes_recovered":131096,"num_keys_recovered":0,"read_bytes_sec":170,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":248},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:36:30.165 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T12:36:30.165 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:30.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:30.267+0000 7f837c591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:30.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:30.274+0000 7f837c591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:30.462 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:30.463 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:30.475 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2024-09-15T12:36:30.203124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185467456,"num_objects":2825,"num_object_clones":0,"num_object_copies":8475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":45,"num_write":7954,"num_write_kb":343380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7935,"ondisk_log_size":7935,"up":75,"acting":75,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214776,"kb_used_data":42616,"kb_used_omap":0,"kb_used_meta":172096,"kb_avail":377272584,"statfs":{"total":386547056640,"available":386327126016,"internally_reserved":0,"allocated":43638784,"data_stored":620713220,"data_compressed":29385610,"data_compressed_allocated":38682624,"data_compressed_original":618676224,"omap_allocated":0,"internal_metadata":176226304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-97779688,"num_objects":-1491,"num_object_clones":0,"num_object_copies":-4473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1493,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":131096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.086132"},"pg_stats":[{"pgid":"7.f","version":"148'523","reported_seq":720,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.550626+0000","last_change":"2024-09-15T12:36:13.515356+0000","last_active":"2024-09-15T12:36:15.550626+0000","last_peered":"2024-09-15T12:36:15.550626+0000","last_clean":"2024-09-15T12:36:15.550626+0000","last_became_active":"2024-09-15T12:36:05.995918+0000","last_became_peered":"2024-09-15T12:36:05.995918+0000","last_unstale":"2024-09-15T12:36:15.550626+0000","last_undegraded":"2024-09-15T12:36:15.550626+0000","last_fullsized":"2024-09-15T12:36:15.550626+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":523,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"140'310","reported_seq":347,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.533850+0000","last_change":"2024-09-15T12:34:43.579635+0000","last_active":"2024-09-15T12:34:58.533850+0000","last_peered":"2024-09-15T12:34:58.533850+0000","last_clean":"2024-09-15T12:34:58.533850+0000","last_became_active":"2024-09-15T12:34:43.396238+0000","last_became_peered":"2024-09-15T12:34:43.396238+0000","last_unstale":"2024-09-15T12:36:01.764535+0000","last_undegraded":"2024-09-15T12:34:58.533850+0000","last_fullsized":"2024-09-15T12:34:58.533850+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":310,"num_object_clones":0,"num_object_copies":930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":19840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":196608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"149'636","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.279506+0000","last_change":"2024-09-15T12:36:10.041250+0000","last_active":"2024-09-15T12:36:26.279506+0000","last_peered":"2024-09-15T12:36:26.279506+0000","last_clean":"2024-09-15T12:36:26.279506+0000","last_became_active":"2024-09-15T12:36:05.959397+0000","last_became_peered":"2024-09-15T12:36:05.959397+0000","last_unstale":"2024-09-15T12:36:26.279506+0000","last_undegraded":"2024-09-15T12:36:26.279506+0000","last_fullsized":"2024-09-15T12:36:26.279506+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3342360,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":636,"num_write_kb":21953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":65560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.c","version":"149'629","reported_seq":716,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.386200+0000","last_change":"2024-09-15T12:36:10.492150+0000","last_active":"2024-09-15T12:36:26.386200+0000","last_peered":"2024-09-15T12:36:26.386200+0000","last_clean":"2024-09-15T12:36:26.386200+0000","last_became_active":"2024-09-15T12:36:05.941034+0000","last_became_peered":"2024-09-15T12:36:05.941034+0000","last_unstale":"2024-09-15T12:36:26.386200+0000","last_undegraded":"2024-09-15T12:36:26.386200+0000","last_fullsized":"2024-09-15T12:36:26.386200+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":629,"num_write_kb":21504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"148'491","reported_seq":711,"reported_epoch":148,"state":"active+recovering+degraded","last_fresh":"2024-09-15T12:36:15.611672+0000","last_change":"2024-09-15T12:36:09.103977+0000","last_active":"2024-09-15T12:36:15.611672+0000","last_peered":"2024-09-15T12:36:15.611672+0000","last_clean":"2024-09-15T12:34:58.587173+0000","last_became_active":"2024-09-15T12:36:05.989572+0000","last_became_peered":"2024-09-15T12:36:05.989572+0000","last_unstale":"2024-09-15T12:36:15.611672+0000","last_undegraded":"2024-09-15T12:36:05.895696+0000","last_fullsized":"2024-09-15T12:36:15.611672+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":491,"num_write_kb":22400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"0,1,3","objects":204},{"shards":"1,3","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"148'411","reported_seq":565,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.438511+0000","last_change":"2024-09-15T12:35:48.808494+0000","last_active":"2024-09-15T12:36:15.438511+0000","last_peered":"2024-09-15T12:36:15.438511+0000","last_clean":"2024-09-15T12:36:15.438511+0000","last_became_active":"2024-09-15T12:35:47.887493+0000","last_became_peered":"2024-09-15T12:35:47.887493+0000","last_unstale":"2024-09-15T12:36:15.438511+0000","last_undegraded":"2024-09-15T12:36:15.438511+0000","last_fullsized":"2024-09-15T12:36:15.438511+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10944512,"num_objects":167,"num_object_clones":0,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":411,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3014656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"149'611","reported_seq":696,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.272815+0000","last_change":"2024-09-15T12:36:09.540255+0000","last_active":"2024-09-15T12:36:26.272815+0000","last_peered":"2024-09-15T12:36:26.272815+0000","last_clean":"2024-09-15T12:36:26.272815+0000","last_became_active":"2024-09-15T12:36:05.980983+0000","last_became_peered":"2024-09-15T12:36:05.980983+0000","last_unstale":"2024-09-15T12:36:26.272815+0000","last_undegraded":"2024-09-15T12:36:26.272815+0000","last_fullsized":"2024-09-15T12:36:26.272815+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2818048,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"140'305","reported_seq":338,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.636058+0000","last_change":"2024-09-15T12:34:43.517925+0000","last_active":"2024-09-15T12:34:58.636058+0000","last_peered":"2024-09-15T12:34:58.636058+0000","last_clean":"2024-09-15T12:34:58.636058+0000","last_became_active":"2024-09-15T12:34:43.358315+0000","last_became_peered":"2024-09-15T12:34:43.358315+0000","last_unstale":"2024-09-15T12:36:01.764537+0000","last_undegraded":"2024-09-15T12:34:58.636058+0000","last_fullsized":"2024-09-15T12:34:58.636058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988480,"num_objects":305,"num_object_clones":0,"num_object_copies":915,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"148'492","reported_seq":693,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.570505+0000","last_change":"2024-09-15T12:36:07.374317+0000","last_active":"2024-09-15T12:36:15.570505+0000","last_peered":"2024-09-15T12:36:15.570505+0000","last_clean":"2024-09-15T12:36:15.570505+0000","last_became_active":"2024-09-15T12:36:05.944934+0000","last_became_peered":"2024-09-15T12:36:05.944934+0000","last_unstale":"2024-09-15T12:36:15.570505+0000","last_undegraded":"2024-09-15T12:36:15.570505+0000","last_fullsized":"2024-09-15T12:36:15.570505+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12845056,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":492,"num_write_kb":22016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":3276800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"140'307","reported_seq":334,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.595593+0000","last_change":"2024-09-15T12:34:10.905570+0000","last_active":"2024-09-15T12:34:58.595593+0000","last_peered":"2024-09-15T12:34:58.595593+0000","last_clean":"2024-09-15T12:34:58.595593+0000","last_became_active":"2024-09-15T12:34:10.888754+0000","last_became_peered":"2024-09-15T12:34:10.888754+0000","last_unstale":"2024-09-15T12:36:01.764543+0000","last_undegraded":"2024-09-15T12:34:58.595593+0000","last_fullsized":"2024-09-15T12:34:58.595593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20119552,"num_objects":307,"num_object_clones":0,"num_object_copies":921,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":307,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.4","version":"149'611","reported_seq":780,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.250859+0000","last_change":"2024-09-15T12:35:48.902283+0000","last_active":"2024-09-15T12:36:26.250859+0000","last_peered":"2024-09-15T12:36:26.250859+0000","last_clean":"2024-09-15T12:36:26.250859+0000","last_became_active":"2024-09-15T12:35:47.879055+0000","last_became_peered":"2024-09-15T12:35:47.879055+0000","last_unstale":"2024-09-15T12:36:26.250859+0000","last_undegraded":"2024-09-15T12:36:26.250859+0000","last_fullsized":"2024-09-15T12:36:26.250859+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2686976,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":611,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"140'314","reported_seq":342,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:58.667827+0000","last_change":"2024-09-15T12:34:43.399802+0000","last_active":"2024-09-15T12:34:58.667827+0000","last_peered":"2024-09-15T12:34:58.667827+0000","last_clean":"2024-09-15T12:34:58.667827+0000","last_became_active":"2024-09-15T12:34:43.394245+0000","last_became_peered":"2024-09-15T12:34:43.394245+0000","last_unstale":"2024-09-15T12:36:01.764547+0000","last_undegraded":"2024-09-15T12:34:58.667827+0000","last_fullsized":"2024-09-15T12:34:58.667827+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":314,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":314,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"148'471","reported_seq":605,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:16.995834+0000","last_change":"2024-09-15T12:36:08.724927+0000","last_active":"2024-09-15T12:36:16.995834+0000","last_peered":"2024-09-15T12:36:16.995834+0000","last_clean":"2024-09-15T12:36:16.995834+0000","last_became_active":"2024-09-15T12:36:05.941883+0000","last_became_peered":"2024-09-15T12:36:05.941883+0000","last_unstale":"2024-09-15T12:36:16.995834+0000","last_undegraded":"2024-09-15T12:36:16.995834+0000","last_fullsized":"2024-09-15T12:36:16.995834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11337728,"num_objects":173,"num_object_clones":0,"num_object_copies":519,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":131072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"149'712","reported_seq":803,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.230995+0000","last_change":"2024-09-15T12:36:09.002505+0000","last_active":"2024-09-15T12:36:26.230995+0000","last_peered":"2024-09-15T12:36:26.230995+0000","last_clean":"2024-09-15T12:36:26.230995+0000","last_became_active":"2024-09-15T12:36:05.965072+0000","last_became_peered":"2024-09-15T12:36:05.965072+0000","last_unstale":"2024-09-15T12:36:26.230995+0000","last_undegraded":"2024-09-15T12:36:26.230995+0000","last_fullsized":"2024-09-15T12:36:26.230995+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":23936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"148'537","reported_seq":712,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.510327+0000","last_change":"2024-09-15T12:35:49.003540+0000","last_active":"2024-09-15T12:36:15.510327+0000","last_peered":"2024-09-15T12:36:15.510327+0000","last_clean":"2024-09-15T12:36:15.510327+0000","last_became_active":"2024-09-15T12:35:47.873977+0000","last_became_peered":"2024-09-15T12:35:47.873977+0000","last_unstale":"2024-09-15T12:36:15.510327+0000","last_undegraded":"2024-09-15T12:36:15.510327+0000","last_fullsized":"2024-09-15T12:36:15.510327+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":537,"num_write_kb":23872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"148'520","reported_seq":718,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:15.420717+0000","last_change":"2024-09-15T12:36:14.870404+0000","last_active":"2024-09-15T12:36:15.420717+0000","last_peered":"2024-09-15T12:36:15.420717+0000","last_clean":"2024-09-15T12:36:15.420717+0000","last_became_active":"2024-09-15T12:36:05.958461+0000","last_became_peered":"2024-09-15T12:36:05.958461+0000","last_unstale":"2024-09-15T12:36:15.420717+0000","last_undegraded":"2024-09-15T12:36:15.420717+0000","last_fullsized":"2024-09-15T12:36:15.420717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:34:09.748678+0000","last_clean_scrub_stamp":"2024-09-15T12:34:09.748678+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14286848,"num_objects":218,"num_object_clones":0,"num_object_copies":654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":520,"num_write_kb":23616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":184877080,"num_objects":2822,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7876,"num_write_kb":342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":552,"num_bytes_recovered":25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":37810176,"data_stored":604831792,"data_compressed":28719722,"data_compressed_allocated":37801984,"data_compressed_original":604831744,"omap_allocated":0,"internal_metadata":0},"log_size":7880,"ondisk_log_size":7880,"up":48,"acting":48,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8028160,"data_stored":128450560,"data_compressed":6099520,"data_compressed_allocated":8028160,"data_compressed_original":128450560,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15785984,"data_stored":252575744,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":24510488,"data_compressed":1163888,"data_compressed_allocated":1531904,"data_compressed_original":24510464,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12460032,"data_stored":199295000,"data_compressed":9463592,"data_compressed_allocated":12455936,"data_compressed_original":199294976,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:31.436 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T12:36:31.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:31.546+0000 7f4506991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:31.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:31.551+0000 7f4506991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:32.130 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-15T12:36:32.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-15T12:36:32.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:32.236+0000 7f8778191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:32.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:32.249+0000 7f8778191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:33.145 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'unique_pool_4' removed 2024-09-15T12:36:33.168 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T12:36:33.177 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T12:36:33.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:33.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:33.580+0000 7facc1a74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:33.590 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:33.587+0000 7facc1a74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:33.781 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:33.781 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:33.797 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":582,"stamp":"2024-09-15T12:36:33.768994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214776,"kb_used_data":42616,"kb_used_omap":0,"kb_used_meta":172096,"kb_avail":377272584,"statfs":{"total":386547056640,"available":386327126016,"internally_reserved":0,"allocated":43638784,"data_stored":620713220,"data_compressed":29385610,"data_compressed_allocated":38682624,"data_compressed_original":618676224,"omap_allocated":0,"internal_metadata":176226304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-184877080,"num_objects":-2822,"num_object_clones":0,"num_object_copies":-8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-7876,"num_write_kb":-342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-552,"num_bytes_recovered":-25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.638572"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:33.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T12:36:33.906 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:33.901+0000 7f5969b48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:33.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:33.908+0000 7f5969b48640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:34.167 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:34.168 INFO:teuthology.orchestra.run.smithi103.stdout:{"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5/8475 objects degraded (0.059%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":880,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":1},"osdmap":{"epoch":154,"num_osds":4,"num_up_osds":4,"osd_up_since":1726403764,"num_in_osds":4,"osd_in_since":1726402915,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"stale+active+clean","count":7},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":25,"num_pools":3,"num_objects":2825,"data_bytes":185467456,"bytes_used":219930624,"bytes_avail":386327126016,"bytes_total":386547056640,"degraded_objects":5,"degraded_total":8475,"degraded_ratio":0.00058997050147492625,"recovering_objects_per_sec":6,"recovering_bytes_per_sec":21844,"recovering_keys_per_sec":0,"num_objects_recovered":40,"num_bytes_recovered":131096,"num_keys_recovered":0,"read_bytes_sec":170,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":248},"fsmap":{"epoch":1,"btime":"2024-09-15T12:21:53:636321+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-15T12:23:00.011054+0000","services":{}},"progress_events":{}} 2024-09-15T12:36:34.181 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-15T12:36:34.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:34.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:34.282+0000 7fdceaf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:34.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:34.290+0000 7fdceaf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:34.477 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:34.477 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:34.489 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":583,"stamp":"2024-09-15T12:36:34.204085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":214776,"kb_used_data":42616,"kb_used_omap":0,"kb_used_meta":172096,"kb_avail":377272584,"statfs":{"total":386547056640,"available":386327126016,"internally_reserved":0,"allocated":43638784,"data_stored":620713220,"data_compressed":29385610,"data_compressed_allocated":38682624,"data_compressed_original":618676224,"omap_allocated":0,"internal_metadata":176226304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-184877080,"num_objects":-2822,"num_object_clones":0,"num_object_copies":-8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-7876,"num_write_kb":-342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-552,"num_bytes_recovered":-25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001390"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.789622+0000","last_change":"2024-09-15T12:34:43.797240+0000","last_active":"2024-09-15T12:34:43.789622+0000","last_peered":"2024-09-15T12:34:43.789622+0000","last_clean":"2024-09-15T12:34:43.789622+0000","last_became_active":"2024-09-15T12:34:43.365163+0000","last_became_peered":"2024-09-15T12:34:43.365163+0000","last_unstale":"2024-09-15T12:36:01.764540+0000","last_undegraded":"2024-09-15T12:34:43.789622+0000","last_fullsized":"2024-09-15T12:34:43.789622+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":295,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794603+0000","last_change":"2024-09-15T12:34:04.808722+0000","last_active":"2024-09-15T12:34:43.794603+0000","last_peered":"2024-09-15T12:34:43.794603+0000","last_clean":"2024-09-15T12:34:43.794603+0000","last_became_active":"2024-09-15T12:34:04.039721+0000","last_became_peered":"2024-09-15T12:34:04.039721+0000","last_unstale":"2024-09-15T12:36:01.764545+0000","last_undegraded":"2024-09-15T12:34:43.794603+0000","last_fullsized":"2024-09-15T12:34:43.794603+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":296,"reported_epoch":140,"state":"stale+active+clean","last_fresh":"2024-09-15T12:34:43.794627+0000","last_change":"2024-09-15T12:34:43.801867+0000","last_active":"2024-09-15T12:34:43.794627+0000","last_peered":"2024-09-15T12:34:43.794627+0000","last_clean":"2024-09-15T12:34:43.794627+0000","last_became_active":"2024-09-15T12:34:43.380238+0000","last_became_peered":"2024-09-15T12:34:43.380238+0000","last_unstale":"2024-09-15T12:36:01.764549+0000","last_undegraded":"2024-09-15T12:34:43.794627+0000","last_fullsized":"2024-09-15T12:34:43.794627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":128,"seq":549755813947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56832,"kb_used_data":16560,"kb_used_omap":0,"kb_used_meta":40256,"kb_avail":94315008,"statfs":{"total":96636764160,"available":96578568192,"internally_reserved":0,"allocated":16957440,"data_stored":253050269,"data_compressed":11992722,"data_compressed_allocated":15785984,"data_compressed_original":252575744,"omap_allocated":0,"internal_metadata":41222144},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:37.491 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:37.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:37.605+0000 7f4b37ac8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:37.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:37.613+0000 7f4b37ac8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:37.799 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:37.799 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:37.812 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":584,"stamp":"2024-09-15T12:36:36.204507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202292,"kb_used_data":27316,"kb_used_omap":0,"kb_used_meta":174912,"kb_avail":377285068,"statfs":{"total":386547056640,"available":386339909632,"internally_reserved":0,"allocated":27971584,"data_stored":368184999,"data_compressed":17392888,"data_compressed_allocated":22896640,"data_compressed_original":366100480,"omap_allocated":0,"internal_metadata":179109888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-184877080,"num_objects":-2822,"num_object_clones":0,"num_object_copies":-8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-7876,"num_write_kb":-342273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-552,"num_bytes_recovered":-25886744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.436005"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":337,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.799572+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:05.799572+0000","last_peered":"2024-09-15T12:36:05.799572+0000","last_clean":"2024-09-15T12:36:05.799572+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:05.799572+0000","last_undegraded":"2024-09-15T12:36:05.799572+0000","last_fullsized":"2024-09-15T12:36:05.799572+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":332,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-15T12:36:33.791006+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:33.791006+0000","last_peered":"2024-09-15T12:36:33.791006+0000","last_clean":"2024-09-15T12:36:33.791006+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:33.791006+0000","last_undegraded":"2024-09-15T12:36:33.791006+0000","last_fullsized":"2024-09-15T12:36:33.791006+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":392,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.912929+0000","last_change":"2024-09-15T12:36:05.914442+0000","last_active":"2024-09-15T12:36:05.912929+0000","last_peered":"2024-09-15T12:36:05.912929+0000","last_clean":"2024-09-15T12:34:03.904193+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:05.912929+0000","last_undegraded":"2024-09-15T12:36:05.912929+0000","last_fullsized":"2024-09-15T12:36:05.912929+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":291,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-15T12:36:33.788955+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:33.788955+0000","last_peered":"2024-09-15T12:36:33.788955+0000","last_clean":"2024-09-15T12:36:33.788955+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:33.788955+0000","last_undegraded":"2024-09-15T12:36:33.788955+0000","last_fullsized":"2024-09-15T12:36:33.788955+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":394,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.904285+0000","last_change":"2024-09-15T12:36:05.906536+0000","last_active":"2024-09-15T12:36:05.904285+0000","last_peered":"2024-09-15T12:36:05.904285+0000","last_clean":"2024-09-15T12:22:07.063698+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:05.904285+0000","last_undegraded":"2024-09-15T12:36:05.904285+0000","last_fullsized":"2024-09-15T12:36:05.904285+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.138526+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:26.138526+0000","last_peered":"2024-09-15T12:36:26.138526+0000","last_clean":"2024-09-15T12:36:26.138526+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:26.138526+0000","last_undegraded":"2024-09-15T12:36:26.138526+0000","last_fullsized":"2024-09-15T12:36:26.138526+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":276,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-15T12:36:05.836631+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:05.836631+0000","last_peered":"2024-09-15T12:36:05.836631+0000","last_clean":"2024-09-15T12:36:05.836631+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:05.836631+0000","last_undegraded":"2024-09-15T12:36:05.836631+0000","last_fullsized":"2024-09-15T12:36:05.836631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":415,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-15T12:36:26.142857+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:26.142857+0000","last_peered":"2024-09-15T12:36:26.142857+0000","last_clean":"2024-09-15T12:36:26.142857+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:26.142857+0000","last_undegraded":"2024-09-15T12:36:26.142857+0000","last_fullsized":"2024-09-15T12:36:26.142857+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":332,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-15T12:36:33.790503+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:33.790503+0000","last_peered":"2024-09-15T12:36:33.790503+0000","last_clean":"2024-09-15T12:36:33.790503+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:33.790503+0000","last_undegraded":"2024-09-15T12:36:33.790503+0000","last_fullsized":"2024-09-15T12:36:33.790503+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":66524,"kb_used_data":13900,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94305316,"statfs":{"total":96636764160,"available":96568643584,"internally_reserved":0,"allocated":14233600,"data_stored":208055856,"data_compressed":9857752,"data_compressed_allocated":12976128,"data_compressed_original":207536128,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860715,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":42328,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":94329512,"statfs":{"total":96636764160,"available":96593420288,"internally_reserved":0,"allocated":2957312,"data_stored":27441932,"data_compressed":1281080,"data_compressed_allocated":1687552,"data_compressed_original":26918912,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192542,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44348,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327492,"statfs":{"total":96636764160,"available":96591351808,"internally_reserved":0,"allocated":1290240,"data_stored":522048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49092,"kb_used_data":9268,"kb_used_omap":0,"kb_used_meta":39808,"kb_avail":94322748,"statfs":{"total":96636764160,"available":96586493952,"internally_reserved":0,"allocated":9490432,"data_stored":132165163,"data_compressed":6254056,"data_compressed_allocated":8232960,"data_compressed_original":131645440,"omap_allocated":0,"internal_metadata":40763392},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:37.812 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T12:36:37.812 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T12:36:37.813 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-15T12:36:37.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-15T12:36:37.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:37.917+0000 7f04834d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:37.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:37.926+0000 7f04834d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:37.944 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:37.945 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:37.948 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:37.948 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:37.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-15T12:36:38.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.060+0000 7f3b1a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.069+0000 7f3b1a191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:38.088 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:38.091 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:38.092 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:38.092 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-15T12:36:38.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.195+0000 7fa7f2391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.204+0000 7fa7f2391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.223 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:38.224 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:38.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:38.227 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:38.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-15T12:36:38.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.329+0000 7fbcf8646640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:38.338+0000 7fbcf8646640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:no valid command found; 10 closest matches: 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:bench [] [] [] [] 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:config get 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:config help 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:config set ... 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:config show 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:deep_scrub 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_ops 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:dump_historic_slow_ops 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:dump_metrics [] 2024-09-15T12:36:38.357 INFO:teuthology.orchestra.run.smithi103.stderr:dump_ops_in_flight 2024-09-15T12:36:38.358 INFO:teuthology.orchestra.run.smithi103.stderr:Error EINVAL: invalid command 2024-09-15T12:36:38.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T12:36:38.360 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-15T12:36:43.361 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T12:36:43.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:43.477+0000 7f74766bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:43.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:43.483+0000 7f74766bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:43.499 INFO:teuthology.orchestra.run.smithi103.stdout:{} 2024-09-15T12:36:43.500 INFO:teuthology.orchestra.run.smithi103.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T12:36:43.511 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T12:36:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:43.613+0000 7f5149b51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:43.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:43.623+0000 7f5149b51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:44.772 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T12:36:44.794 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T12:36:44.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:44.899+0000 7f11e2097640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:44.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:44.909+0000 7f11e2097640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:45.776 INFO:teuthology.orchestra.run.smithi103.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T12:36:45.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T12:36:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:45.904+0000 7f17f00ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:45.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:45.917+0000 7f17f00ac640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:46.777 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T12:36:46.798 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T12:36:46.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:46.903+0000 7f645aabd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:46.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:46.913+0000 7f645aabd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:47.783 INFO:teuthology.orchestra.run.smithi103.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T12:36:47.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T12:36:47.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:47.909+0000 7f1413b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:47.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:47.919+0000 7f1413b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:48.788 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T12:36:48.810 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T12:36:48.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:48.912+0000 7f0e2a148640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:48.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:48.923+0000 7f0e2a148640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:49.793 INFO:teuthology.orchestra.run.smithi103.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T12:36:49.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T12:36:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:49.915+0000 7f666a791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:49.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:49.925+0000 7f666a791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:50.802 INFO:teuthology.orchestra.run.smithi103.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T12:36:50.822 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T12:36:50.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:50.926+0000 7f6c21874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:50.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:50.937+0000 7f6c21874640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:51.803 INFO:teuthology.orchestra.run.smithi103.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T12:36:51.824 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T12:36:51.824 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T12:36:51.824 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:36:51.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:51.936+0000 7fc473a74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:51.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:51.942+0000 7fc473a74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.133 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:52.133 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":162,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:36:51.801263+0000","last_up_change":"2024-09-15T12:36:04.771680+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T12:22:03.901604+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":138,"last_clean_end":140,"up_from":143,"up_thru":147,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4255374138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4255374138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4255374138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4255374138}]},"public_addr":"172.21.15.103:6801/4255374138","cluster_addr":"172.21.15.103:6800/4255374138","heartbeat_back_addr":"172.21.15.103:6806/4255374138","heartbeat_front_addr":"172.21.15.103:6805/4255374138","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":128,"last_clean_end":142,"up_from":147,"up_thru":147,"down_at":145,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2321609765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":2321609765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":2321609765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2321609765}]},"public_addr":"172.21.15.103:6804/2321609765","cluster_addr":"172.21.15.103:6807/2321609765","heartbeat_back_addr":"172.21.15.103:6803/2321609765","heartbeat_front_addr":"172.21.15.103:6808/2321609765","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":53,"last_clean_end":94,"up_from":96,"up_thru":147,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":1936795873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1936795873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":1936795873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1936795873}]},"public_addr":"172.21.15.117:6803/1936795873","cluster_addr":"172.21.15.117:6802/1936795873","heartbeat_back_addr":"172.21.15.117:6807/1936795873","heartbeat_front_addr":"172.21.15.117:6806/1936795873","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":78,"last_clean_end":92,"up_from":99,"up_thru":147,"down_at":93,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":752053507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":752053507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":752053507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":752053507}]},"public_addr":"172.21.15.117:6810/752053507","cluster_addr":"172.21.15.117:6809/752053507","heartbeat_back_addr":"172.21.15.117:6818/752053507","heartbeat_front_addr":"172.21.15.117:6817/752053507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-15T12:35:26.761592+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-15T12:36:01.762572+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-15T12:31:25.715879+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-15T12:30:54.724533+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:36:52.147 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T12:36:52.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:36:52.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.250+0000 7fb3d4951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.257+0000 7fb3d4951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.446 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:52.446 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":162,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:36:51.801263+0000","last_up_change":"2024-09-15T12:36:04.771680+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T12:22:03.901604+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":138,"last_clean_end":140,"up_from":143,"up_thru":147,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4255374138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4255374138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4255374138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4255374138}]},"public_addr":"172.21.15.103:6801/4255374138","cluster_addr":"172.21.15.103:6800/4255374138","heartbeat_back_addr":"172.21.15.103:6806/4255374138","heartbeat_front_addr":"172.21.15.103:6805/4255374138","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":128,"last_clean_end":142,"up_from":147,"up_thru":147,"down_at":145,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2321609765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":2321609765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":2321609765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2321609765}]},"public_addr":"172.21.15.103:6804/2321609765","cluster_addr":"172.21.15.103:6807/2321609765","heartbeat_back_addr":"172.21.15.103:6803/2321609765","heartbeat_front_addr":"172.21.15.103:6808/2321609765","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":53,"last_clean_end":94,"up_from":96,"up_thru":147,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":1936795873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1936795873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":1936795873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1936795873}]},"public_addr":"172.21.15.117:6803/1936795873","cluster_addr":"172.21.15.117:6802/1936795873","heartbeat_back_addr":"172.21.15.117:6807/1936795873","heartbeat_front_addr":"172.21.15.117:6806/1936795873","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":78,"last_clean_end":92,"up_from":99,"up_thru":147,"down_at":93,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":752053507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":752053507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":752053507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":752053507}]},"public_addr":"172.21.15.117:6810/752053507","cluster_addr":"172.21.15.117:6809/752053507","heartbeat_back_addr":"172.21.15.117:6818/752053507","heartbeat_front_addr":"172.21.15.117:6817/752053507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-15T12:35:26.761592+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-15T12:36:01.762572+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-15T12:31:25.715879+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-15T12:30:54.724533+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:36:52.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T12:36:52.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T12:36:52.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T12:36:52.461 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T12:36:52.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.572+0000 7f8c20108640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.572+0000 7f5f1044f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.580 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.572+0000 7f1544191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.572+0000 7fdfb1488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.582+0000 7f1544191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.583+0000 7f8c20108640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.583+0000 7f5f1044f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.584+0000 7fdfb1488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.619 INFO:teuthology.orchestra.run.smithi103.stdout:425201762611 2024-09-15T12:36:52.619 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T12:36:52.620 INFO:teuthology.orchestra.run.smithi103.stdout:412316860741 2024-09-15T12:36:52.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T12:36:52.622 INFO:teuthology.orchestra.run.smithi103.stdout:631360192559 2024-09-15T12:36:52.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T12:36:52.624 INFO:teuthology.orchestra.run.smithi103.stdout:614180323393 2024-09-15T12:36:52.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T12:36:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.726+0000 7fe3b6391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.726+0000 7f242ac7f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.736+0000 7f242ac7f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.736+0000 7fe3b6391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.776+0000 7f5305b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.784+0000 7f5305b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.786+0000 7f768fef4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:52.797+0000 7f768fef4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:52.949 INFO:teuthology.orchestra.run.smithi103.stdout:425201762604 2024-09-15T12:36:52.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 425201762611 got 425201762604 for osd.3 2024-09-15T12:36:52.972 INFO:teuthology.orchestra.run.smithi103.stdout:412316860715 2024-09-15T12:36:52.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 412316860741 got 412316860715 for osd.2 2024-09-15T12:36:52.995 INFO:teuthology.orchestra.run.smithi103.stdout:631360192542 2024-09-15T12:36:53.002 INFO:teuthology.orchestra.run.smithi103.stdout:614180323388 2024-09-15T12:36:53.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192559 got 631360192542 for osd.1 2024-09-15T12:36:53.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 614180323393 got 614180323388 for osd.0 2024-09-15T12:36:53.964 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T12:36:53.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T12:36:54.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T12:36:54.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T12:36:54.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.079+0000 7fee2f24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.091+0000 7fee2f24f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.125+0000 7fe0010b4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.125+0000 7f8efa908640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.136+0000 7fe0010b4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.137+0000 7f8efa908640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.164+0000 7fd099791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:54.173+0000 7fd099791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:54.333 INFO:teuthology.orchestra.run.smithi103.stdout:425201762604 2024-09-15T12:36:54.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 425201762611 got 425201762604 for osd.3 2024-09-15T12:36:54.347 INFO:teuthology.orchestra.run.smithi103.stdout:412316860715 2024-09-15T12:36:54.356 INFO:teuthology.orchestra.run.smithi103.stdout:631360192542 2024-09-15T12:36:54.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 412316860741 got 412316860715 for osd.2 2024-09-15T12:36:54.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192559 got 631360192542 for osd.1 2024-09-15T12:36:54.384 INFO:teuthology.orchestra.run.smithi103.stdout:614180323388 2024-09-15T12:36:54.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 614180323393 got 614180323388 for osd.0 2024-09-15T12:36:55.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T12:36:55.362 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T12:36:55.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T12:36:55.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T12:36:55.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.461+0000 7ff5a5b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.473+0000 7ff5a5b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.509 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.502+0000 7f24bdf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.505+0000 7f42458ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.513+0000 7f24bdf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.516+0000 7f42458ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.548+0000 7fda0a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.557+0000 7fda0a991640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.722 INFO:teuthology.orchestra.run.smithi103.stdout:412316860741 2024-09-15T12:36:55.731 INFO:teuthology.orchestra.run.smithi103.stdout:425201762611 2024-09-15T12:36:55.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 412316860741 got 412316860741 for osd.2 2024-09-15T12:36:55.740 DEBUG:teuthology.parallel:result is None 2024-09-15T12:36:55.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 425201762611 got 425201762611 for osd.3 2024-09-15T12:36:55.745 DEBUG:teuthology.parallel:result is None 2024-09-15T12:36:55.756 INFO:teuthology.orchestra.run.smithi103.stdout:614180323393 2024-09-15T12:36:55.764 INFO:teuthology.orchestra.run.smithi103.stdout:631360192559 2024-09-15T12:36:55.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 614180323393 got 614180323393 for osd.0 2024-09-15T12:36:55.769 DEBUG:teuthology.parallel:result is None 2024-09-15T12:36:55.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 631360192559 got 631360192559 for osd.1 2024-09-15T12:36:55.778 DEBUG:teuthology.parallel:result is None 2024-09-15T12:36:55.778 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T12:36:55.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:55.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.922+0000 7f3c9c151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:55.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:55.930+0000 7f3c9c151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.115 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:56.115 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:56.128 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2024-09-15T12:36:54.208855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.420782"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:56.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:56.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.232+0000 7f3fa6b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.239+0000 7f3fa6b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:56.424 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:56.438 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2024-09-15T12:36:56.209339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416632"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:56.439 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T12:36:56.439 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T12:36:56.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.541+0000 7f592c0a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.552+0000 7f592c0a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.759 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T12:36:56.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T12:36:56.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:56.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.870+0000 7f420b0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:56.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:56.877+0000 7f420b0d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.063 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:57.063 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:57.076 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2024-09-15T12:36:56.209339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416632"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:57.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:57.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.180+0000 7f5d3c0c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.188+0000 7f5d3c0c8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.371 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:57.371 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:57.384 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2024-09-15T12:36:56.209339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416632"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:57.385 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T12:36:57.385 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:36:57.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.487+0000 7fef4d591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.495+0000 7fef4d591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.681 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:57.682 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:36:57.694 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":602,"stamp":"2024-09-15T12:36:56.209339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416632"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860741,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:36:57.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T12:36:57.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.798+0000 7fce88951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:57.806+0000 7fce88951640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:57.993 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:36:57.993 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":162,"fsid":"0089f1fe-9ae6-4cff-ad25-a1642089e108","created":"2024-09-15T12:21:53.636578+0000","modified":"2024-09-15T12:36:51.801263+0000","last_up_change":"2024-09-15T12:36:04.771680+0000","last_in_change":"2024-09-15T12:21:55.751033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T12:22:02.016490+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T12:22:03.901604+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"37772177-3b71-4af7-b52e-b013e757f215","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":138,"last_clean_end":140,"up_from":143,"up_thru":147,"down_at":141,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6801","nonce":4255374138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4255374138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4255374138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6805","nonce":4255374138}]},"public_addr":"172.21.15.103:6801/4255374138","cluster_addr":"172.21.15.103:6800/4255374138","heartbeat_back_addr":"172.21.15.103:6806/4255374138","heartbeat_front_addr":"172.21.15.103:6805/4255374138","state":["exists","up"]},{"osd":1,"uuid":"5a23b5b3-b2de-48d3-b668-2ebed6c50787","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":128,"last_clean_end":142,"up_from":147,"up_thru":147,"down_at":145,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2321609765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6807","nonce":2321609765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6803","nonce":2321609765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2321609765}]},"public_addr":"172.21.15.103:6804/2321609765","cluster_addr":"172.21.15.103:6807/2321609765","heartbeat_back_addr":"172.21.15.103:6803/2321609765","heartbeat_front_addr":"172.21.15.103:6808/2321609765","state":["exists","up"]},{"osd":2,"uuid":"97c17cfc-5763-433b-b175-3e1ba3a87f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":53,"last_clean_end":94,"up_from":96,"up_thru":147,"down_at":95,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6803","nonce":1936795873}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1936795873}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6807","nonce":1936795873}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1936795873}]},"public_addr":"172.21.15.117:6803/1936795873","cluster_addr":"172.21.15.117:6802/1936795873","heartbeat_back_addr":"172.21.15.117:6807/1936795873","heartbeat_front_addr":"172.21.15.117:6806/1936795873","state":["exists","up"]},{"osd":3,"uuid":"502045c7-d23c-4241-9fbd-971c75c81272","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":78,"last_clean_end":92,"up_from":99,"up_thru":147,"down_at":93,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":752053507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6809","nonce":752053507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":752053507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6817","nonce":752053507}]},"public_addr":"172.21.15.117:6810/752053507","cluster_addr":"172.21.15.117:6809/752053507","heartbeat_back_addr":"172.21.15.117:6818/752053507","heartbeat_front_addr":"172.21.15.117:6817/752053507","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-15T12:35:26.761592+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"2024-09-15T12:36:01.762572+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"2024-09-15T12:31:25.715879+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"2024-09-15T12:30:54.724533+0000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T12:36:59.008 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T12:36:59.008 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T12:36:59.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.120+0000 7fd6d4911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.133+0000 7fd6d4911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.150 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:36:59.150 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "" 2024-09-15T12:36:59.150 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:36:59.162 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T12:36:59.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.264+0000 7fc0bf6bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.273+0000 7fc0bf6bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.460 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T12:36:59.478 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T12:36:59.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T12:36:59.591 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.581+0000 7f82b9111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.594 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.593+0000 7f82b9111640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.610 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:36:59.611 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "" 2024-09-15T12:36:59.611 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:36:59.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T12:36:59.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.725+0000 7f53e2191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:36:59.734+0000 7f53e2191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:36:59.931 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T12:36:59.953 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T12:36:59.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T12:37:00.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.058+0000 7f7c82151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.070+0000 7f7c82151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.086 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:37:00.087 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "" 2024-09-15T12:37:00.087 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:37:00.098 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T12:37:00.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.200+0000 7f9d7b548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.209+0000 7f9d7b548640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.398 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T12:37:00.414 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T12:37:00.414 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T12:37:00.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.517+0000 7f2823191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.529+0000 7f2823191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.547 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-15T12:37:00.547 INFO:teuthology.orchestra.run.smithi103.stdout: "success": "" 2024-09-15T12:37:00.547 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-15T12:37:00.558 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T12:37:00.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.661+0000 7f250e8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.670+0000 7f250e8bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.859 INFO:teuthology.orchestra.run.smithi103.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T12:37:00.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:37:00.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.976+0000 7fcd74f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:00.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:00.984+0000 7fcd74f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:01.170 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:37:01.170 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:37:01.183 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":604,"stamp":"2024-09-15T12:37:00.210311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.406329"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":379,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.815942+0000","last_change":"2024-09-15T12:35:47.863753+0000","last_active":"2024-09-15T12:36:51.815942+0000","last_peered":"2024-09-15T12:36:51.815942+0000","last_clean":"2024-09-15T12:36:51.815942+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:36:51.815942+0000","last_undegraded":"2024-09-15T12:36:51.815942+0000","last_fullsized":"2024-09-15T12:36:51.815942+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:02.016608+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818655+0000","last_change":"2024-09-15T12:36:26.181523+0000","last_active":"2024-09-15T12:36:51.818655+0000","last_peered":"2024-09-15T12:36:51.818655+0000","last_clean":"2024-09-15T12:36:51.818655+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:51.818655+0000","last_undegraded":"2024-09-15T12:36:51.818655+0000","last_fullsized":"2024-09-15T12:36:51.818655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":435,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819688+0000","last_change":"2024-09-15T12:36:26.153173+0000","last_active":"2024-09-15T12:36:51.819688+0000","last_peered":"2024-09-15T12:36:51.819688+0000","last_clean":"2024-09-15T12:36:51.819688+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:51.819688+0000","last_undegraded":"2024-09-15T12:36:51.819688+0000","last_fullsized":"2024-09-15T12:36:51.819688+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":315,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819754+0000","last_change":"2024-09-15T12:36:26.184075+0000","last_active":"2024-09-15T12:36:51.819754+0000","last_peered":"2024-09-15T12:36:51.819754+0000","last_clean":"2024-09-15T12:36:51.819754+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:51.819754+0000","last_undegraded":"2024-09-15T12:36:51.819754+0000","last_fullsized":"2024-09-15T12:36:51.819754+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817399+0000","last_change":"2024-09-15T12:36:26.147093+0000","last_active":"2024-09-15T12:36:51.817399+0000","last_peered":"2024-09-15T12:36:51.817399+0000","last_clean":"2024-09-15T12:36:51.817399+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:51.817399+0000","last_undegraded":"2024-09-15T12:36:51.817399+0000","last_fullsized":"2024-09-15T12:36:51.817399+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":318,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.818264+0000","last_change":"2024-09-15T12:36:01.819152+0000","last_active":"2024-09-15T12:36:51.818264+0000","last_peered":"2024-09-15T12:36:51.818264+0000","last_clean":"2024-09-15T12:36:51.818264+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:36:51.818264+0000","last_undegraded":"2024-09-15T12:36:51.818264+0000","last_fullsized":"2024-09-15T12:36:51.818264+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":356,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.819227+0000","last_change":"2024-09-15T12:36:26.181229+0000","last_active":"2024-09-15T12:36:51.819227+0000","last_peered":"2024-09-15T12:36:51.819227+0000","last_clean":"2024-09-15T12:36:51.819227+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:51.819227+0000","last_undegraded":"2024-09-15T12:36:51.819227+0000","last_fullsized":"2024-09-15T12:36:51.819227+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762611,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323393,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:37:01.187 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T12:22:02.016608+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=2, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.187 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:01.188 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T12:37:21.189 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:37:21.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:21.307+0000 7f65c2d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:21.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:21.315+0000 7f65c2d91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:21.500 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:37:21.501 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:37:21.513 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2024-09-15T12:37:20.215270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003077"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":381,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.860781+0000","last_change":"2024-09-15T12:37:00.898339+0000","last_active":"2024-09-15T12:37:00.860781+0000","last_peered":"2024-09-15T12:37:00.860781+0000","last_clean":"2024-09-15T12:37:00.860781+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:37:00.860781+0000","last_undegraded":"2024-09-15T12:37:00.860781+0000","last_fullsized":"2024-09-15T12:37:00.860781+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"11'52","last_scrub_stamp":"2024-09-15T12:37:00.897955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.897955+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":358,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.934209+0000","last_change":"2024-09-15T12:36:59.968414+0000","last_active":"2024-09-15T12:36:59.934209+0000","last_peered":"2024-09-15T12:36:59.934209+0000","last_clean":"2024-09-15T12:36:59.934209+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:59.934209+0000","last_undegraded":"2024-09-15T12:36:59.934209+0000","last_fullsized":"2024-09-15T12:36:59.934209+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2024-09-15T12:36:59.968208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.968208+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.462323+0000","last_change":"2024-09-15T12:36:59.477794+0000","last_active":"2024-09-15T12:36:59.462323+0000","last_peered":"2024-09-15T12:36:59.462323+0000","last_clean":"2024-09-15T12:36:59.462323+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:59.462323+0000","last_undegraded":"2024-09-15T12:36:59.462323+0000","last_fullsized":"2024-09-15T12:36:59.462323+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.477610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.477610+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":317,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.932834+0000","last_change":"2024-09-15T12:36:59.964847+0000","last_active":"2024-09-15T12:36:59.932834+0000","last_peered":"2024-09-15T12:36:59.932834+0000","last_clean":"2024-09-15T12:36:59.932834+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:59.932834+0000","last_undegraded":"2024-09-15T12:36:59.932834+0000","last_fullsized":"2024-09-15T12:36:59.932834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.964671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.964671+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":439,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.461346+0000","last_change":"2024-09-15T12:36:59.473906+0000","last_active":"2024-09-15T12:36:59.461346+0000","last_peered":"2024-09-15T12:36:59.461346+0000","last_clean":"2024-09-15T12:36:59.461346+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:59.461346+0000","last_undegraded":"2024-09-15T12:36:59.461346+0000","last_fullsized":"2024-09-15T12:36:59.461346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2024-09-15T12:36:59.473715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.473715+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.817216+0000","last_change":"2024-09-15T12:36:01.794213+0000","last_active":"2024-09-15T12:36:51.817216+0000","last_peered":"2024-09-15T12:36:51.817216+0000","last_clean":"2024-09-15T12:36:51.817216+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:36:51.817216+0000","last_undegraded":"2024-09-15T12:36:51.817216+0000","last_fullsized":"2024-09-15T12:36:51.817216+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":320,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.862291+0000","last_change":"2024-09-15T12:37:00.919755+0000","last_active":"2024-09-15T12:37:00.862291+0000","last_peered":"2024-09-15T12:37:00.862291+0000","last_clean":"2024-09-15T12:37:00.862291+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:37:00.862291+0000","last_undegraded":"2024-09-15T12:37:00.862291+0000","last_fullsized":"2024-09-15T12:37:00.862291+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:37:00.919578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.919578+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":454,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:51.814931+0000","last_change":"2024-09-15T12:36:01.791797+0000","last_active":"2024-09-15T12:36:51.814931+0000","last_peered":"2024-09-15T12:36:51.814931+0000","last_clean":"2024-09-15T12:36:51.814931+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:36:51.814931+0000","last_undegraded":"2024-09-15T12:36:51.814931+0000","last_fullsized":"2024-09-15T12:36:51.814931+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:22:04.028340+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":358,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.934947+0000","last_change":"2024-09-15T12:36:59.970946+0000","last_active":"2024-09-15T12:36:59.934947+0000","last_peered":"2024-09-15T12:36:59.934947+0000","last_clean":"2024-09-15T12:36:59.934947+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:59.934947+0000","last_undegraded":"2024-09-15T12:36:59.934947+0000","last_fullsized":"2024-09-15T12:36:59.934947+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.970764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.970764+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762634,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860745,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:37:21.514 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:21.514 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T12:22:04.028340+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=22, tm_sec=4, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=12, tm_min=36, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T12:37:21.515 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T12:37:41.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T12:37:41.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:41.630+0000 7f07bd191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:41.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:41.638+0000 7f07bd191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:41.825 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-15T12:37:41.826 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-15T12:37:41.838 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":624,"stamp":"2024-09-15T12:37:40.220191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590376,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":78,"num_write_kb":1107,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":377487360,"kb_used":202656,"kb_used_data":5472,"kb_used_omap":0,"kb_used_meta":197120,"kb_avail":377284704,"statfs":{"total":386547056640,"available":386339536896,"internally_reserved":0,"allocated":5603328,"data_stored":3952781,"data_compressed":90168,"data_compressed_allocated":122880,"data_compressed_original":1720320,"omap_allocated":0,"internal_metadata":201850880},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002909"},"pg_stats":[{"pgid":"1.0","version":"11'52","reported_seq":381,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.860781+0000","last_change":"2024-09-15T12:37:00.898339+0000","last_active":"2024-09-15T12:37:00.860781+0000","last_peered":"2024-09-15T12:37:00.860781+0000","last_clean":"2024-09-15T12:37:00.860781+0000","last_became_active":"2024-09-15T12:35:47.855206+0000","last_became_peered":"2024-09-15T12:35:47.855206+0000","last_unstale":"2024-09-15T12:37:00.860781+0000","last_undegraded":"2024-09-15T12:37:00.860781+0000","last_fullsized":"2024-09-15T12:37:00.860781+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"11'52","last_scrub_stamp":"2024-09-15T12:37:00.897955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.897955+0000","last_clean_scrub_stamp":"2024-09-15T12:22:02.016608+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":358,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.934209+0000","last_change":"2024-09-15T12:36:59.968414+0000","last_active":"2024-09-15T12:36:59.934209+0000","last_peered":"2024-09-15T12:36:59.934209+0000","last_clean":"2024-09-15T12:36:59.934209+0000","last_became_active":"2024-09-15T12:36:05.998878+0000","last_became_peered":"2024-09-15T12:36:05.998878+0000","last_unstale":"2024-09-15T12:36:59.934209+0000","last_undegraded":"2024-09-15T12:36:59.934209+0000","last_fullsized":"2024-09-15T12:36:59.934209+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2024-09-15T12:36:59.968208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.968208+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":437,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.462323+0000","last_change":"2024-09-15T12:36:59.477794+0000","last_active":"2024-09-15T12:36:59.462323+0000","last_peered":"2024-09-15T12:36:59.462323+0000","last_clean":"2024-09-15T12:36:59.462323+0000","last_became_active":"2024-09-15T12:36:05.906548+0000","last_became_peered":"2024-09-15T12:36:05.906548+0000","last_unstale":"2024-09-15T12:36:59.462323+0000","last_undegraded":"2024-09-15T12:36:59.462323+0000","last_fullsized":"2024-09-15T12:36:59.462323+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.477610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.477610+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":317,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.932834+0000","last_change":"2024-09-15T12:36:59.964847+0000","last_active":"2024-09-15T12:36:59.932834+0000","last_peered":"2024-09-15T12:36:59.932834+0000","last_clean":"2024-09-15T12:36:59.932834+0000","last_became_active":"2024-09-15T12:36:05.963470+0000","last_became_peered":"2024-09-15T12:36:05.963470+0000","last_unstale":"2024-09-15T12:36:59.932834+0000","last_undegraded":"2024-09-15T12:36:59.932834+0000","last_fullsized":"2024-09-15T12:36:59.932834+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.964671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.964671+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":439,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.461346+0000","last_change":"2024-09-15T12:36:59.473906+0000","last_active":"2024-09-15T12:36:59.461346+0000","last_peered":"2024-09-15T12:36:59.461346+0000","last_clean":"2024-09-15T12:36:59.461346+0000","last_became_active":"2024-09-15T12:36:05.897768+0000","last_became_peered":"2024-09-15T12:36:05.897768+0000","last_unstale":"2024-09-15T12:36:59.461346+0000","last_undegraded":"2024-09-15T12:36:59.461346+0000","last_fullsized":"2024-09-15T12:36:59.461346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2024-09-15T12:36:59.473715+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.473715+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":1,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":456,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.399372+0000","last_change":"2024-09-15T12:37:00.411858+0000","last_active":"2024-09-15T12:37:00.399372+0000","last_peered":"2024-09-15T12:37:00.399372+0000","last_clean":"2024-09-15T12:37:00.399372+0000","last_became_active":"2024-09-15T12:35:47.849162+0000","last_became_peered":"2024-09-15T12:35:47.849162+0000","last_unstale":"2024-09-15T12:37:00.399372+0000","last_undegraded":"2024-09-15T12:37:00.399372+0000","last_fullsized":"2024-09-15T12:37:00.399372+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:37:00.411661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.411661+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":320,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.862291+0000","last_change":"2024-09-15T12:37:00.919755+0000","last_active":"2024-09-15T12:37:00.862291+0000","last_peered":"2024-09-15T12:37:00.862291+0000","last_clean":"2024-09-15T12:37:00.862291+0000","last_became_active":"2024-09-15T12:35:47.876938+0000","last_became_peered":"2024-09-15T12:35:47.876938+0000","last_unstale":"2024-09-15T12:37:00.862291+0000","last_undegraded":"2024-09-15T12:37:00.862291+0000","last_fullsized":"2024-09-15T12:37:00.862291+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:37:00.919578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.919578+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":456,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:37:00.400389+0000","last_change":"2024-09-15T12:37:00.413646+0000","last_active":"2024-09-15T12:37:00.400389+0000","last_peered":"2024-09-15T12:37:00.400389+0000","last_clean":"2024-09-15T12:37:00.400389+0000","last_became_active":"2024-09-15T12:35:47.855772+0000","last_became_peered":"2024-09-15T12:35:47.855772+0000","last_unstale":"2024-09-15T12:37:00.400389+0000","last_undegraded":"2024-09-15T12:37:00.400389+0000","last_fullsized":"2024-09-15T12:37:00.400389+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:37:00.413428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:37:00.413428+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":358,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-15T12:36:59.934947+0000","last_change":"2024-09-15T12:36:59.970946+0000","last_active":"2024-09-15T12:36:59.934947+0000","last_peered":"2024-09-15T12:36:59.934947+0000","last_clean":"2024-09-15T12:36:59.934947+0000","last_became_active":"2024-09-15T12:36:05.989886+0000","last_became_peered":"2024-09-15T12:36:05.989886+0000","last_unstale":"2024-09-15T12:36:59.934947+0000","last_undegraded":"2024-09-15T12:36:59.934947+0000","last_fullsized":"2024-09-15T12:36:59.934947+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T12:36:59.970764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T12:36:59.970764+0000","last_clean_scrub_stamp":"2024-09-15T12:22:04.028340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":8,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":2,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":44,"num_write":77,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1180736,"data_compressed":60112,"data_compressed_allocated":81920,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":99,"seq":425201762634,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49844,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":48448,"kb_avail":94321996,"statfs":{"total":96636764160,"available":96585723904,"internally_reserved":0,"allocated":1413120,"data_stored":1135773,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49610752},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":96,"seq":412316860775,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58872,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":147,"seq":631360192602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44412,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":43072,"kb_avail":94327428,"statfs":{"total":96636764160,"available":96591286272,"internally_reserved":0,"allocated":1355776,"data_stored":545424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44105728},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":143,"seq":614180323418,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":49528,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":48128,"kb_avail":94322312,"statfs":{"total":96636764160,"available":96586047488,"internally_reserved":0,"allocated":1417216,"data_stored":1135792,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":49283072},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":590368,"data_compressed":30056,"data_compressed_allocated":40960,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T12:37:41.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T12:37:41.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:41.943+0000 7f5dbfd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:41.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-15T12:37:41.954+0000 7f5dbfd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-15T12:37:42.173 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T12:37:42.173 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T12:37:42.173 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T12:37:42.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.257 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T12:37:42.258 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T12:37:42.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.353 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T12:37:42.353 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T12:37:42.353 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.443 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T12:37:42.443 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T12:37:42.444 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.539 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T12:37:42.539 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T12:37:42.539 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T12:37:42.539 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.567 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T12:37:42.567 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T12:37:42.568 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T12:37:42.568 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T12:37:42.580 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T12:37:42.580 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T12:37:42.581 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-15T12:37:42.658 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:37:42.658 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T12:37:42.810 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:37:42.811 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T12:37:42.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:37:42.889 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T12:37:42.978 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:37:42.979 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T12:37:43.063 INFO:tasks.ceph:Archiving mon data... 2024-09-15T12:37:43.064 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/data/mon.a.tgz 2024-09-15T12:37:43.065 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-15T12:37:43.080 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.QrWOJrz3il 2024-09-15T12:37:43.080 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QrWOJrz3il 2024-09-15T12:37:43.260 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.QrWOJrz3il 2024-09-15T12:37:43.343 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.QrWOJrz3il is 281KB 2024-09-15T12:37:43.436 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.QrWOJrz3il 2024-09-15T12:37:43.451 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T12:37:43.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T12:37:43.492 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T12:37:43.569 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T12:37:43.585 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T12:37:43.585 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T12:37:43.586 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi103/crash 2024-09-15T12:37:43.587 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T12:37:43.634 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi117/crash 2024-09-15T12:37:43.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T12:37:43.682 INFO:tasks.ceph:Compressing logs... 2024-09-15T12:37:43.682 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T12:37:43.685 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T12:37:43.707 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T12:37:43.707 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39119.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28708.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28628.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5/var/log/ceph/ceph-client.admin.28708.log: --verbose -- /var/log/ceph/ceph-client.admin.28668.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28708.log.gz 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.28628.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28628.log.gz 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-client.admin.28668.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28668.log.gz 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28579.log 2024-09-15T12:37:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.28579.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28579.log.gz 2024-09-15T12:37:43.799 INFO:teuthology.orchestra.run.smithi117.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50967.log 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.39119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46217.log 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39119.log.gz 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33559.log 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50967.log.gz 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37693.log 2024-09-15T12:37:43.889 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46217.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46426.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33559.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50043.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37693.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31339.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46426.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34865.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50043.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29313.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31339.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34865.log.gz 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-09-15T12:37:43.890 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29313.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35953.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41707.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35953.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43610.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40718.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.41707.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42360.log 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43610.log.gz 2024-09-15T12:37:43.891 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40718.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39837.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42360.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39298.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31419.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39837.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32174.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.39298.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51345.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31419.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38899.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32174.log.gz 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33608.log 2024-09-15T12:37:43.892 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51345.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38899.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33608.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33510.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29759.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33510.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31247.log 2024-09-15T12:37:43.893 INFO:teuthology.orchestra.run.smithi103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29759.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32948.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37377.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31247.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32948.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33414.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37377.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40984.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33414.log.gz 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50841.log 2024-09-15T12:37:43.894 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 47.6% -- replaced with /var/log/ceph/ceph-client.admin.40984.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50841.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50460.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46524.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46266.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50460.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46524.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47570.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46266.log.gz 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42487.log 2024-09-15T12:37:43.895 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41525.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47570.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44447.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42487.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41525.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35798.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44447.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32997.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.35798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32997.log.gz 2024-09-15T12:37:43.896 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34814.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29483.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51051.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34814.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31833.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29483.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30004.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51051.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31833.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31833.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46018.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30004.log.gz 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30278.log 2024-09-15T12:37:43.897 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40852.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30278.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42741.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37511.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40852.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37511.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37511.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32221.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51303.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47664.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31560.log 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32221.log.gz 2024-09-15T12:37:43.898 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51303.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51303.log.gz -5 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30142.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49665.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31560.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31560.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30142.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30142.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47265.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49665.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49665.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30002.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45874.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47265.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30002.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45874.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45874.log.gz 2024-09-15T12:37:43.899 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51429.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30575.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45694.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51429.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30575.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30575.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37602.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36396.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48182.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31207.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-09-15T12:37:43.900 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48182.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31207.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41210.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51261.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49462.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33777.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51261.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51261.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43919.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49462.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50925.log 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33777.log.gz 2024-09-15T12:37:43.901 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49378.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43919.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43919.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50925.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38591.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49378.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45917.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38591.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37082.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29315.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34216.log 2024-09-15T12:37:43.902 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37082.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37082.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29315.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44705.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34216.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31966.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.44705.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30764.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43741.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31966.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31966.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37153.log 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30764.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30764.log.gz 2024-09-15T12:37:43.903 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47404.log: 49.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35647.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:gzip/var/log/ceph/ceph-client.admin.37153.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37153.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41971.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35647.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33046.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33657.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41971.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48462.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33046.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33657.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33657.log.gz 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-09-15T12:37:43.904 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28718.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48462.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48462.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30896.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.28718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28718.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30896.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30896.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37964.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31022.log 2024-09-15T12:37:43.905 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37964.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37964.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33186.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38125.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31022.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31022.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36836.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49917.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38125.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28947.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36836.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36836.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49917.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49917.log.gz 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49161.log 2024-09-15T12:37:43.906 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.28947.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28947.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48851.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36929.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49161.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38766.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48851.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48851.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36929.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48631.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38766.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50145.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47530.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-09-15T12:37:43.907 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48631.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50145.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34404.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50001.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35471.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35083.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35221.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41301.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29171.log 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35471.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35471.log.gz 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41301.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41301.log.gzgzip 2024-09-15T12:37:43.908 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49076.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37651.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29171.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.29171.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32529.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.49076.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37651.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37826.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32529.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42868.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37826.log.gz 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28690.log 2024-09-15T12:37:43.909 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42868.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46088.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.28690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34589.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28690.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49336.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46088.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34589.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33737.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49336.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46749.log 2024-09-15T12:37:43.910 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33737.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31919.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46749.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30100.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40188.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31919.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31919.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38055.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29843.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30100.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30100.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31620.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38055.log.gz 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44229.log 2024-09-15T12:37:43.911 INFO:teuthology.orchestra.run.smithi103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.29843.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31620.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31620.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44229.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29423.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47490.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29423.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29423.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48720.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50799.log 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47490.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47490.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48720.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48720.log.gz 2024-09-15T12:37:43.912 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49582.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50799.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50799.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39214.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39032.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49582.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49582.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35123.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39214.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39032.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr: 48.0% -- replaced with /var/log/ceph/ceph-client.admin.39032.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39928.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35123.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37306.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39928.log.gz 2024-09-15T12:37:43.913 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41931.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37306.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42527.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.46798.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41931.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41931.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr: 88.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46798.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46798.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.39256.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42064.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39256.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39256.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38411.log 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-09-15T12:37:43.914 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38273.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38411.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38411.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38273.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.38273.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35607.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49833.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37469.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29312.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35607.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31167.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49833.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47314.log 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29312.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31167.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31167.log.gz 2024-09-15T12:37:43.915 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49203.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50313.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47314.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40902.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49203.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29314.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50313.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32401.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40902.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34934.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29314.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.32401.log.gz 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33866.log 2024-09-15T12:37:43.916 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34934.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41027.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42961.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33866.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41027.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41027.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29591.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42961.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29675.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41434.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29591.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29591.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51177.log 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29675.log.gz 2024-09-15T12:37:43.917 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32422.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41434.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34040.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51177.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51177.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32422.log: gzip -5 49.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32422.log.gz /var/log/ceph/ceph-client.admin.44496.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33137.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34040.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44496.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34547.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33137.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33137.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2024-09-15T12:37:43.918 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48551.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34547.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45469.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42193.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42313.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48551.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39970.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45469.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51135.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42313.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43830.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39970.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51135.log.gz 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-09-15T12:37:43.919 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43830.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43830.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38233.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29011.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33088.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38233.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38233.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45048.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29011.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36355.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33088.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39746.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45048.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50589.log 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36355.log.gz 2024-09-15T12:37:43.920 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39746.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50589.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50589.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32033.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46130.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32033.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32033.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34081.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31771.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.46130.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45097.log 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34081.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34081.log.gz 2024-09-15T12:37:43.921 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49026.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31771.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45097.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36488.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49026.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49026.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49502.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35688.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36488.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49502.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49502.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35904.log 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35688.log.gz 2024-09-15T12:37:43.922 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30713.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33697.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35904.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35904.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37286.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30713.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42801.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.33697.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32879.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37286.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37286.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30833.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42801.log.gz 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32879.log: gzip -5 50.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.32879.log.gz -- 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.50273.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44754.log 2024-09-15T12:37:43.923 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30833.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50273.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44754.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44754.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50312.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37917.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41797.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37917.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.37917.log.gz -5 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr: --verbose -- /var/log/ceph/ceph-client.admin.29482.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43529.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46160.log 2024-09-15T12:37:43.924 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29482.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31071.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.46160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43782.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31071.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31120.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43782.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46306.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31706.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31120.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.31120.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46306.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46306.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35381.log 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31706.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.31706.log.gz 2024-09-15T12:37:43.925 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35381.log: gzip -5 --verbose -- 50.4% -- replaced with /var/log/ceph/ceph-client.admin.35381.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.49119.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43437.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49119.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49119.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.41658.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39886.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38724.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41658.log: gzip -5 --verbose 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41658.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.38806.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38724.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38724.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30229.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38806.log: gzip -5 49.6% --verbose -- /var/log/ceph/ceph-client.admin.31379.log 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-09-15T12:37:43.926 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35341.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30229.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47617.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31379.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32576.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35341.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42106.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47617.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39522.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32576.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49791.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42106.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39522.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.35728.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39522.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49791.log.gz 2024-09-15T12:37:43.927 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38104.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35728.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48591.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45371.log: /var/log/ceph/ceph-client.admin.38104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32357.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38104.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48591.log: gzip 49.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.48591.log.gz --verbose 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.38362.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:gzip/var/log/ceph/ceph-client.admin.32357.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32357.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38362.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.38362.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 49.2% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.928 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50421.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38849.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42614.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38849.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50650.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40323.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50650.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50650.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40323.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36265.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40323.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29087.log 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2024-09-15T12:37:43.929 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36265.log: -- /var/log/ceph/ceph-client.admin.38631.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36265.log.gz 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29087.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47185.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29087.log.gz 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36443.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38631.log.gz 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47185.log: -- /var/log/ceph/ceph-client.admin.45146.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47185.log.gz/var/log/ceph/ceph-client.admin.36443.log: 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29481.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36443.log.gz 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5/var/log/ceph/ceph-client.admin.45146.log: --verbose -- /var/log/ceph/ceph-client.admin.38990.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45146.log.gz/var/log/ceph/ceph-client.admin.29481.log: 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29481.log.gz 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38990.log: gzip/var/log/ceph/ceph-client.admin.46887.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29717.log 2024-09-15T12:37:43.930 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0%/var/log/ceph/ceph-client.admin.49542.log: -- replaced with /var/log/ceph/ceph-client.admin.38990.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43486.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29717.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34633.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36754.log: /var/log/ceph/ceph-client.admin.43486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44545.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43486.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34633.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34633.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43569.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44545.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44545.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42701.log 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43569.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43569.log.gz 2024-09-15T12:37:43.931 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33817.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42701.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42701.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29905.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33817.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33817.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29905.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29905.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35492.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43243.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44380.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35492.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35492.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41250.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44380.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44380.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35261.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41250.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41250.log.gz 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2024-09-15T12:37:43.932 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49622.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35261.log: /var/log/ceph/ceph-client.admin.38549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35261.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49622.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49622.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42233.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43149.log 49.2% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42233.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42233.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46847.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.48671.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46847.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46847.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34502.log 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36572.log 49.2% -- replaced with /var/log/ceph/ceph-client.admin.48671.log.gz 2024-09-15T12:37:43.933 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34502.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34502.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44312.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36572.log: gzip -5 50.0% --verbose -- /var/log/ceph/ceph-client.admin.32127.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36572.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40103.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44312.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5/var/log/ceph/ceph-client.admin.32127.log: --verbose -- /var/log/ceph/ceph-client.admin.40514.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.32127.log.gz/var/log/ceph/ceph-client.admin.40103.log: 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40103.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40944.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40514.log: /var/log/ceph/ceph-client.admin.40494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48342.log 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40514.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-09-15T12:37:43.934 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40944.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40944.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34453.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48342.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48342.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30673.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51009.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30673.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34453.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30673.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51009.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.51009.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30533.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44095.log: gzip -5 --verbose -- 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.48511.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30533.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.30533.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43196.log 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48511.log: gzip -5 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-15T12:37:43.935 INFO:teuthology.orchestra.run.smithi103.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30982.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43196.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43196.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30982.log: gzip 50.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30982.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:gzip/var/log/ceph/ceph-client.admin.38191.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49294.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40627.log: -- /var/log/ceph/ceph-client.admin.40148.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49294.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49420.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49294.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40148.log.gz 2024-09-15T12:37:43.936 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49420.log: -- /var/log/ceph/ceph.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49420.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33461.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39389.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32268.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39389.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39389.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32268.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32268.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51093.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40676.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51093.log: /var/log/ceph/ceph-client.admin.51511.log: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51093.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39165.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49959.log 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40676.log: 88.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T12:37:43.937 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39165.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40676.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27376.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39165.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46927.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49959.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph.tmp-client.admin.27376.log: -- /var/log/ceph/ceph-client.admin.48231.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27376.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46927.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46927.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36530.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48231.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48413.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36530.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36530.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48413.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34263.log 2024-09-15T12:37:43.938 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35301.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33915.log: /var/log/ceph/ceph-client.admin.34263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34263.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35301.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50482.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35301.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44665.log: -- /var/log/ceph/ceph-client.admin.28454.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50482.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50482.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.28454.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.28454.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45977.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37020.log: /var/log/ceph/ceph-client.admin.45977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43008.log 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-09-15T12:37:43.939 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45977.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34774.log: gzip -5 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43008.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43008.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39438.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34732.log: /var/log/ceph/ceph-client.admin.29801.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz/var/log/ceph/ceph-client.admin.39438.log: 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37062.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39438.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29801.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37062.log: -- /var/log/ceph/ceph-client.admin.50272.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37062.log.gz 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.46386.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50883.log 2024-09-15T12:37:43.940 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50272.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50883.log: -- /var/log/ceph/ceph-client.admin.32315.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50883.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36663.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39480.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32315.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.32315.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49707.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39480.log: gzip -5 --verbose -- 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39480.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.42447.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49707.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49707.log.gz 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30003.log 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42447.log: /var/log/ceph/ceph-client.admin.43339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 49.2% 2024-09-15T12:37:43.941 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42447.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30003.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30003.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41392.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41392.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41392.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37735.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48902.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31465.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37735.log.gz 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51471.log 2024-09-15T12:37:43.942 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48986.log: /var/log/ceph/ceph-client.admin.31465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38682.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31465.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29129.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51471.log: /var/log/ceph/ceph-client.admin.38682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30624.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51471.log.gz -- replaced with /var/log/ceph/ceph-client.admin.38682.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29129.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29129.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36883.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30624.log: gzip -5 --verbose 50.4% -- replaced with /var/log/ceph/ceph-client.admin.30624.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr: -- /var/log/ceph/ceph-client.admin.32674.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36883.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36883.log.gz 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.32472.log 2024-09-15T12:37:43.943 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32674.log: gzip -5 49.6% --verbose -- /var/log/ceph/ceph-client.admin.29213.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32674.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.32472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41616.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42567.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29213.log: /var/log/ceph/ceph-client.admin.41616.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29213.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41616.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48066.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42567.log: /var/log/ceph/ceph-client.admin.48066.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42567.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48066.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41126.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41474.log 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42273.log: /var/log/ceph/ceph-client.admin.41126.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-09-15T12:37:43.944 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41126.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42661.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41474.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41474.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43388.log: 48.8% -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33960.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36313.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33960.log: /var/log/ceph/ceph-client.admin.42661.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.33960.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42661.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43872.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36313.log: 50.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40280.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36313.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49252.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43872.log: /var/log/ceph/ceph-client.admin.40280.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.43872.log.gz 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40280.log.gz 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50461.log 2024-09-15T12:37:43.945 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49252.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41077.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49252.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44179.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50461.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50461.log.gz/var/log/ceph/ceph-client.admin.41077.log: 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41077.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43290.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51219.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44179.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44179.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.43290.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43290.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32625.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51219.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51219.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32625.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.32625.log.gz 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39662.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28533.log 2024-09-15T12:37:43.946 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38006.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39662.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39662.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39562.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34169.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.28533.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28533.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39562.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39562.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49875.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29633.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34169.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34169.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.49875.log: 49.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49875.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34122.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.29633.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29633.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47445.log: 49.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38458.log 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-09-15T12:37:43.947 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50143.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34122.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34122.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.38458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38458.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42022.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44803.log: 49.6% 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50143.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34310.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42022.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35997.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34310.log: /var/log/ceph/ceph-client.admin.45420.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34310.log.gz 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51387.log 2024-09-15T12:37:43.948 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35997.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44585.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.51387.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.51387.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31293.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36592.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.31293.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.31293.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.36592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36592.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48280.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-09-15T12:37:43.949 INFO:teuthology.orchestra.run.smithi103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41889.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30172.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35566.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41889.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41889.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34000.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34000.log: /var/log/ceph/ceph-client.0.30172.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.34000.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.42153.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30484.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.34996.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35855.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30484.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30484.log.gz 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30813.log 2024-09-15T12:37:43.950 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.35855.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.35855.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37202.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30813.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30813.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48320.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37202.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37202.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48320.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48320.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47056.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45743.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37777.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47056.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47056.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37777.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37777.log.gz 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37870.log 2024-09-15T12:37:43.951 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.48113.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.37870.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37870.log.gz 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41350.log 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.50757.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40346.log 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.41350.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41350.log.gz 2024-09-15T12:37:43.952 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2024-09-15T12:37:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.0.40346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45957.log 2024-09-15T12:37:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.40061.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2024-09-15T12:37:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2024-09-15T12:37:43.956 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.45957.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45957.log.gz 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32080.log 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.39613.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47225.log 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.32080.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.32080.log.gz 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30939.log 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.47225.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47225.log.gz 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44026.log 2024-09-15T12:37:43.957 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.30939.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.30939.log.gz 2024-09-15T12:37:43.961 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.44026.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44026.log.gz 2024-09-15T12:37:44.239 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.30172.log.gz 2024-09-15T12:37:44.250 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.46160.log.gz 2024-09-15T12:37:44.322 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.35798.log.gz 2024-09-15T12:37:44.362 INFO:teuthology.orchestra.run.smithi103.stderr: 89.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T12:37:44.367 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.32472.log.gz 2024-09-15T12:37:44.609 INFO:teuthology.orchestra.run.smithi103.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.40346.log.gz 2024-09-15T12:37:58.353 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T12:37:59.470 INFO:teuthology.orchestra.run.smithi117.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T12:37:59.472 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-15T12:37:59.472 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m15.773s 2024-09-15T12:37:59.472 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m29.365s 2024-09-15T12:37:59.472 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m1.079s 2024-09-15T12:38:00.174 INFO:teuthology.orchestra.run.smithi103.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T12:38:00.339 INFO:teuthology.orchestra.run.smithi103.stderr: 92.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T12:38:00.342 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-15T12:38:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m16.644s 2024-09-15T12:38:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m35.091s 2024-09-15T12:38:00.342 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m1.546s 2024-09-15T12:38:00.342 INFO:tasks.ceph:Archiving logs... 2024-09-15T12:38:00.343 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi103/log 2024-09-15T12:38:00.344 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T12:38:05.923 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi117/log 2024-09-15T12:38:05.924 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T12:38:10.769 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T12:38:10.778 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T12:38:10.778 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T12:38:10.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T12:38:10.875 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T12:38:10.883 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T12:38:10.884 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T12:38:10.886 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T12:38:10.901 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-15T12:38:10.901 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 +454us[ +451us] +/- 45ms 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -578us[ -582us] +/- 55ms 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +15us[ +11us] +/- 79ms 2024-09-15T12:38:10.906 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:38:10.907 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T12:38:10.907 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-15T12:38:10.907 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +478us[ +472us] +/- 45ms 2024-09-15T12:38:10.908 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -579us[ -579us] +/- 55ms 2024-09-15T12:38:10.908 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +19us[ +19us] +/- 79ms 2024-09-15T12:38:10.908 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T12:38:10.908 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T12:38:10.916 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T12:38:10.916 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T12:38:10.925 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T12:38:11.024 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T12:38:11.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T12:38:11.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:38:11.117 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T12:38:11.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:38:11.144 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T12:38:11.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T12:38:11.161 INFO:teuthology.task.internal:Duration was 1618.426350 seconds 2024-09-15T12:38:11.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T12:38:11.170 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T12:38:11.171 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T12:38:11.173 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T12:38:11.211 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T12:38:11.223 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T12:38:11.484 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T12:38:11.484 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-15T12:38:11.484 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T12:38:11.545 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-15T12:38:11.545 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T12:38:11.567 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T12:38:11.568 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T12:38:11.588 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T12:38:11.634 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T12:38:11.634 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T12:38:11.656 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T12:38:11.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T12:38:11.783 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T12:38:11.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T12:38:11.809 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T12:38:11.833 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T12:38:11.843 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T12:38:11.852 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T12:38:11.872 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-15T12:38:11.897 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-15T12:38:11.930 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T12:38:11.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:38:11.965 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T12:38:11.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T12:38:11.985 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T12:38:11.994 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T12:38:11.995 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi103 2024-09-15T12:38:11.995 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T12:38:12.040 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-09-15_11:59:21-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7905511/remote/smithi117 2024-09-15T12:38:12.040 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T12:38:12.072 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T12:38:12.072 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T12:38:12.077 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T12:38:12.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T12:38:12.132 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T12:38:12.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T12:38:12.142 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T12:38:12.142 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T12:38:12.144 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T12:38:12.156 INFO:teuthology.orchestra.run.smithi103.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 12:38 /home/ubuntu/cephtest 2024-09-15T12:38:12.177 INFO:teuthology.orchestra.run.smithi117.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 12:38 /home/ubuntu/cephtest 2024-09-15T12:38:12.178 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T12:38:12.187 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T12:38:12.207 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/bluestore thrashers/default thrashosds-health workloads/radosbench} duration: 1618.4263503551483 flavor: crimson owner: scheduled_matan@teuthology success: true 2024-09-15T12:38:12.207 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T12:38:12.287 INFO:teuthology.run:pass